Note:

• This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
• It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.

Use Oracle Cloud Infrastructure Site-to-Site VPN service in HA mode with ECMP routing from Linux and Libreswan

Introduction

In today’s interconnected world, ensuring the availability and security of data transmitted across networks is key. To address this critical need, Oracle Cloud offers robust networking capabilities, including the ability to establish highly available IPSec tunnels. In this tutorial, we will explore the concept of high availability IPSec tunnels and guide you through the process of setting up a resilient network architecture in Oracle Cloud using Equal-cost multi-path (ECMP) protocol.

In this tutorial, we will focus on utilizing Oracle Linux, a powerful and secure operating system optimized for Oracle Cloud environments, along with Libreswan, a well-established IPSec client, to establish IPSec tunnels in route-based mode. We will leverage the Dynamic Routing Gateway (DRG) functionality provided by Oracle Cloud Infrastructure (OCI) to enable seamless failover and load balancing between multiple IPSec tunnels.

Objectives

Provide a comprehensive guide to implementing IPSec Tunnels in OCI using ECMP routing protocol for load balancing traffic along them in active/active scenario.

• Understand the basics of Site-To-Site IPSec protocol
• Understand the different IPSec modes: Transport vs Tunnel, Policy based vs Route based
• Configure Libreswan in Oracle Linux with ECMP load balancing
• Configure DRGv2 for redundant IPSec tunnels
• Test tunnels redundancy and load balancing with Iperf3

By following this tutorial, you will gain a comprehensive understanding of IPSec in OCI. You will acquire the skills necessary to effectively interconnect your on-premises infrastructure with OCI through a redundant connection.

Prerequisites

• An active OCI tenancy. You must have the necessary permissions to create and manage network resources in OCI.

• A basic understanding of Linux OS, Networking concepts and OCI. This includes familiarity with fundamental networking concepts such as IP addressing, subnetting, routing, and firewall is essential.

• A basic understanding of Oracle Linux, including how to install and configure. If you are new to Oracle Linux, you may want to go through a basic Oracle Linux tutorial or guide beforehand.

• A virtual cloud network (VCN) and subnet(s) set up in OCI, with appropriate routing rules, Internet Gateway, DRG and security lists configured.

• A good understanding of how to use the OCI Console or OCI CLI to create and manage network resources.

Note: It is recommended that you have a test environment set up in OCI for experimenting with networking configurations and IPSec before implementing them in a Production environment.

What’s IPSec VPN

Internet Protocol security (IPSec) is a framework of open standards for helping to ensure private, secure communications over Internet Protocol (IP) networks through the use of cryptographic security services. IPSec supports network-level data integrity, data confidentiality, data origin authentication, and replay protection. Because IPSec is integrated at the Internet layer (layer 3), it provides security for almost all protocols in the TCP/IP suite, and because IPSec is applied transparently to applications, there is no need to configure separate security for each application that uses TCP/IP.

IPSec helps provide defense-in-depth against network-based attacks from untrusted computers, attacks that can result in the denial-of-service of applications, services, or the network.

• Data corruption
• Data theft
• User-credential theft
• Administrative control of servers, other computers, and the network.

Site-to-Site VPN

A site-to-site IPSec (Internet Protocol Security) VPN, also known as a network-to-network VPN, establishes a secure and encrypted connection between two or more networks over the internet. It enables the secure transmission of data between geographically distributed sites, creating a virtual private network (VPN) that extends the network’s reach beyond its physical boundaries.

In a site-to-site IPSec VPN, the participating networks, typically belonging to different organizations or remote branches of the same organization, are connected via dedicated IPSec tunnels. These tunnels encapsulate and encrypt network traffic, ensuring its confidentiality, integrity, and authenticity while traversing untrusted networks such as the internet.

On the other hand, a point-to-site VPN (P2S) establishes a secure connection between individual client devices and a remote network. Unlike site-to-site VPNs, which connect networks, P2S VPNs enable secure remote access for individual devices to access the network resources. P2S VPNs are commonly used to enable secure access for remote employees, contractors, or mobile users who need to connect to the organization’s network from external locations.

Note: This tutorial scope is limited to Site-to-Site IPSec VPN that is currently the only one supported in OCI DRGv2.

VPN IPSec Tunnels Concepts

IPSec stands for Internet Protocol Security or IP Security. IPSec is a protocol suite that encrypts the entire IP traffic before the packets are transferred from the source node to the destination. IPSec can be configured in two modes:

• Transport Mode: IPSec only encrypts and/or authenticates the actual payload of the packet, and the header information stays intact.

• Tunnel Mode: IPSec encrypts and/or authenticates the entire packet. After encryption, the packet is then encapsulated to form a new UDP IP packet that has different header information.

IPSec VPN site-to-site tunnels offer the following advantages:

• No need to buy dedicated expensive lease lines from one site to another, as public telecommunication lines are used to transmit data.

• The internal IP addresses of both the participating networks and nodes are hidden from external users.

• The entire communication between the source and destination sites is encrypted, significantly lowering the chances of information theft. OCI supports only the tunnel mode of VPN IPSec and is offered as self-service using the web console.

Note: OCI Site-to-Site VPN ONLY supports tunnel mode so that will be the only mode available in OCI.

Architecture

OCI IPSec with ECMP consist of lists which include:

• DRGv2 attached to VCN1 and VCN2 using default route tables and autogenerated route tables RT2 “Autogenerated Drg Route Table for VCN attachments” in region Frankfurt.

• Same DRGv2 with an IPSec connection including two tunnels:

  • tunnel 1, Oracle VPN IP address 193.122.x.x with autogenerated route table RT1 “Autogenerated Drg Route Table for RPC, VC, and IPSec attachments”.
  • tunnel 2, Oracle VPN IP address 158.101.x.x with autogenerated route table RT1 “Autogenerated Drg Route Table for RPC, VC, and IPSec attachments”.

• An Oracle Linux VM as an IPSec client with ECMP support from on-premises, running in private CIDR range 192.168.0.0/16, public IP 143.47.48.219.

• Iperf3 in Oracle Linux in on-premises as client and as server in Oracle OCI.

• A CPE configured as Libreswan with public IP 143.47.x.x.

• ECMP enabled Oracle Linux 8 and OCI side.

Task 1: Configure OCI Settings

For this tutorial, we have created one Oracle Linux 7 VM instance and installed Libreswan 3.25 on it. In order to install Libreswan into Linux, you can follow the following Oracle documentation: Access to Other Clouds with Libreswan. You can install Libreswan in the environment of your choice. For this tutorial, we have chosen another remote region in OCI as the Libreswan client and tunnel initiator.

Once you have installed Libreswan (without configuring it yet), note the public IP of your Linux 7 VM as well as the private IPv4 CIDR range where you installed Libreswan in.

Now, let’s configure the OCI Settings

• Log in to your OCI console and navigate to the Networking tab to create VCN1 and VCN2, as follows: Create Your VCN.

• Create VCN1 with CIDR 10.0.0.0/16 and two subnets: Subnet A 10.0.1.0/24 and subnet B 10.0.0.0/24.

• Create VCN2 with CIDR 172.20.0.0/16 and two subnets: Subnet C 172.20.1.0/24 and subnet D 172.20.2.0/24.
  
  

• Create VMs in the target subnets in OCI in order to test the IPSec tunnels. In this tutorial we have created 3 VMs (origin1, origin2 and origin3).
  
  

• Now let’s create a DRG, under Networking, Customer Connectivity, Dynamic Routing Gateways.
  
  

• Once DRG is created, you will need to create a Virtual Cloud Network attachment to both VCN1 and VCN2.
  
  
  
  

  • Once created, we will have two Network Attachments (one per VCN) connecting both VCNs to the DRG with an AutoGenerated DRG routing table for VCN attachments (RT2 routing table in the main architecture).

  • This routing table will tell DRG where to send/route to the incoming traffic from the just attached VCNs: Any traffic destinated to “Destination CIDR” will be sent/routed to “Next Hop Attachment” as follows:
    
    
    
    

  • Later on, when creating IPSec tunnels, we will have a similar autogenerated routing table for the incoming traffic from on-premises via IPSec tunnel.
    
    

• Now, let’s create the IPSec connectivity. Before creating the IPSec, we will need to create the Customer-Premises Equipment (CPE) which represents the on-premises device connecting to OCI via IPSec, under Networking, Customer connectivity, Customer-premises equipment, Create CPE:
  
  

  • X.X.X.X must be the PUBLIC IP address where your on-premises device is connecting from. Do NOT confuse it with private IP address assigned to the Libreswan Oracle Linux machine you installed in previous steps. Very likely your Oracle Linux VM will be behind NAT (not owning directly the public IP). If you don’t know the public IP address you can always run the following command from Linux console to find out: curl ifconfig.co.

  • For this tutorial, choose Vendor Libreswan version 3.18 or later.

• Go to Networking, Customer connectivity, Site-to-Site VPN, Create IPSec connection and provide the following details.

  Note: You can use the VPN Wizard, but it is out of scope on this tutorial.

  • Name of your IPSec configuration

  • Create in compartment: Your compartment

  • Customer-premises equipment in Your Compartment: Choose the CPE you created in the previous step

    • This CPE is behind a NAT device: This will directly affect how the Internet Key Exchange (IKE) id will be “presented” to OCI from on-premises. Assuming Libreswan is running behind NAT, if you want to use the private IP where Libreswan is running from as IKE ID, mark this option. Otherwise, if you want to use the public IP where Libreswan is connecting from, don’t mark this option. We will configure this IKE id later on from Libreswan configuration.

  • Dynamic routing gateway compartment: Choose the DRG that you configured in previous steps.

  • Routes to your on-premises network: Dynamic routing is out of scope of this tutorial. We will be focusing on ECMP for load balancing and redundancy, using Route Based IPSec configuration. We will add here manually/statically the on-premises routes/CIDRS we want to be able to reach from OCI to on-premises. For this tutorial we will use 192.168.0.0/16 as on-premises CIDR.

    

    • Tunnel 1:

      • Name
      • Provide customer shared secret : You can use your own IKE pre-shared key (password) or leave it blank so OCI will choose one (we will use it later on during Libreswan setup)
      • IKE version: For this Tutorial we will be using IKEv1
      • Routing type: For this Tutorial we will use static routing for Route Based IPSec configuration, so choose “Static routing”
      • IPv4 inside tunnel interface - CPE and Oracle: Leave this option blank
      • IPv6 addressing: Leave it blank as IPv6 is out of scope in this tutorial

    • Tunnel 2: Fill Tunnel 2 exactly the same way as Tunnel 1 (IKE version, Routing type, and so on).

      

      

  • Click Create IPSec connection. If everything goes well, after some minutes you should have an IPSec configuration running with two tunnels for Redundancy.

    

  • Now that we have the DRG and IPSec in place, we need to make sure that every traffic from VCNs in OCI toward on-premises will find its way to the DRG. Once the traffic reaches DRG, it will be tunnel toward on-premises via the two IPSec tunnels recently configured. To do so, we need to add the on-premises CIDR as a route rule for every subnet route table in VCN (we chose default Route table in the Tutorial), target type Dynamic Routing Gateway, Destination type CIDR Block, then add the on-premises route, in this case 192.168.0.0/16.

    

  Note: The reason we need to add this route is that OCI does not have implicit routing for CIDR addresses outside the VCN realm. As a result, we must manually add the CIDR block to ensure that the traffic reaches the DRG.

Task 2: Configure Linux and Libreswan Settings

This part of the tutorial will focus on the Linux OS and Libreswan configuration steps. The Libreswan we previously installed will act as a Site-to-Site Tunnel initiator and OCI DRG as a Tunnel responder.

• Access your Oracle Linux OS via SSH. Elevate your privileges to run important (admin) commands.

  • sudo su: You will become root user.

  • Make sure Libreswan is installed with the correct version: ipsec –version. You should see Libreswan version 3.25 or higher. If not, go to Task 1: Configure OCI Settings and follow instructions to install Libreswan.

  • Go to config folder: cd /etc.

  • Libreswan stores all tunnels configuration in the ipsec.conf file. User your favorite file editor. “vi” is a good option: vi ipsec.conf

    • Libreswan ipsec.conf:

      • conn [TunnelName1}: Tunnel Name

      • type= tunnel (Tunnel mode)

      • authby=secret (the authentication will be using a secret passphrase)

      • pfs=yes (Perfect forward secrecy enabled)

      • keyexchange=ike

      • leftid= n.n.n.n (This is the public IP address from which Libreswan is connecting. It will be used as the IKE ID identifier, unless you have selected the option ‘CPE is behind NAT’ during IPSec creation. In that case, any IP address or fully qualified domain name (FQDN) can be set as the identifier)

      • leftsourceip= m.m.m.m (This is the private IP assigned to Libreswan i.e. 192.168.1.1)

      • leftsubnet= x.x.x.x/x (This is the CIDR block assigned to Libreswan i.e. 192.168.0.0/16)

      • right= y.y.y.y (This is the OCI VPN “Public” IP address assigned to tunnel1 during IPSec creation under Networking, Customer connectivity, Site-to-Site VPN, YourIPSEC)

        

      • rightid= y.y.y.y (This is the IKE ID sent from OCI. Normally is the OCI VPN “Public” IP address, normally the same as right parameter)

      • leftsubnet=0.0.0.0/0 (This is the CIDR block that will be sent toward OCI as part of the SA Traffic Selector negotiation Internet Key Exchange (IKEv2) Protocol 2.2.9. This does NOT mean all traffic from OCI will be routed to on-premises. Instead, what this CIDR block does is determine which specific traffic will be accepted, encrypted, and tunneled through the IPSec tunnel. The IPSec tunnel described here will be represented by a virtual interface called the Virtual Tunnel Interface (VTI), which acts as a virtual network interface (like a virtual network card) representing the tunnel itself. Let’s say the VTI is called vti01. Any traffic that is routed towards the vti01 interface will become part of the IPSec tunnel and be securely sent to the other end of the connection. This approach, where routing decisions are made based on the specific IPSec tunnel and its associated VTI, is known as IPSec route-based routing)

      • rightsubnet=0.0.0.0/0 (The same as above for traffic accepted to OCI from on-premises)

      • mark=n/0xffffffff (*Needed option for use with VTI interfaces in order to mark packets encapsulated within the IPSec tunnel associated to this VTI interface. Needs to be unique across all tunnels. i.e. 5/0xffffffff*)

      • vti-interface=vtinn (*Name of the VTI interface, i.e. vti01*)

      • vti-routing=no (Whether routes should automatically be created into the VTI device. Choose NO since we don’t want to create 0.0.0.0/0 route automatically)

      • encapsulation= yes/auto (yes forces the NAT detection code to lie and tell the remote peer that RFC-3948 encapsulation (ESP in port 4500 UDP packets). Auto it will force NAT autodetection)

      • aggrmode= no

      • ike=aes_cbc256-sha2_384;modp1536 (IKE encryption/authentication algorithm to be used for the connection (phase 1 aka ISAKMP SA).The format is “cipher-hash;modpgroup, cipher-hash;modpgroup, …)

      • esp=aes_gcm256;modp1536 (Specifies the algorithms that will be offered/accepted for a Child SA negotiation. The format for ESP is ENC-AUTH followed by one optional PFSgroup. For instance, “aes_gcm256” or “aes256-sha2_512-dh14” or “aes-sha2_512+sha2_256”)

      • ikev2= no (To use IKEv2, change to ikev2=insist)

  • Go to config folder: cd /etc/ipsec.d.

  • Libreswan stores all tunnels share secrets (passwords) in the shared.secrets file.

    • Libreswan shared.secrets format: leftid right : PSK “secret”

      • leftid: The IKE Id configured in IPSec.conf (leftid). Normally the Libreswan Public IP address.

      • right : The Public IP address of the OCI Tunnel.

      • “secret” : The current tunnel shared secret/password configured in OCI. You can get this info from OCI console under Networking, Customer connectivity, Site-to-Site VPN, YourIPSEC, TunnelName, Tunnel information, Shared secret, Show.

• For this tutorial, we have created the following ipsec.conf and shared.secrets (public IPs are hidden):

  ipsec.conf

  

  shared.secrets

  

• Now, let’s try to establish the two HA tunnels against OCI.

  • Restart Libreswan service: as root, execute ipsec restart . If everything goes well no errors will be shown, otherwise you will see something like this:

    Job for ipsec.service failed because the control process exited with error code. See “systemctl status ipsec.service” and “journalctl -xe” for details.

  • Now let’s check current status of IPSec tunnels: execute ipsec status. You will see plenty of information about the current tunnels configure around ESP(data plane) and IKE(signaling plane), allowed subnets, and so on. For now, we will focus on the connection list.

    000 Connection list:

    000

    000 Total IPSec connections: *loaded 0, active 0

  • Currently there is no tunnel config loaded, neither active.

  • Execute ipsec auto --add ConnName1, i.e. ipsec auto --add home_liftvti.

    • 002 added connection description “home_liftvti” <– Now the connection is “loaded” not yet active. Check it with ipsec status.

  • Execute ipsec auto --up ConnName1 to initiate tunnel 1, i.e. ipsec auto --up home_liftvti. If everything goes well, you will see IPSec SA established tunnel mode, otherwise you will see different IKE/IPSec errors that you will need to troubleshoot.

    

  • Now is turn for the second tunnel: Execute ipsec auto –add ConnName2, i.e. ipsec auto –add home_liftvti2

    • 002 added connection description “home_liftvti2” <– Now the connection is “loaded” not yet active. Check it with ipsec status

  • Execute ipsec auto –up ConnName2 to initiate tunnel 1, i.e. ipsec auto –up home_liftvti2. If everthing goes well you will see “IPSec SA established tunnel mode”, otherwise you will see different IKE/IPSec errors that you will need to troubleshoot

    

Task 3: Configure IP routing and Tunnel traffic

This part of the tutorial will focus on IP routing and Tunnel traffic.

• Now that both tunnels are up and running, let’s try to ping some machine in OCI from Libreswan to test tunnels. For instance, ping origin1 with IP 10.0.0.109. You will see that it will fail.

  PING 10.0.0.109 (10.0.0.109) 56(84) bytes of data.
  
    10.0.0.109 ping statistics
    26 packets transmitted, 0 received, 100% packet loss,
  

• You might be wondering why it’s not working, even though the IPSec tunnels are up and running against OCI. The answer is because the two tunnels that we have established against OCI are “route based” and every tunnel is represented by a Virtual Interface (VTI). You can see every VTI as a virtual network interface card (NIC) that represents an IPSec tunnel. Any traffic routed to the VTI is encapsulated using the ESP (Encapsulating Security Payload) protocol and sent to the other end of the tunnel. The VTI acts as a gateway for traffic that needs to be secured and transmitted through the IPSec tunnel.

• In our testing scenario, we have two VTI interfaces, vti01 and vti02, each of one representing home_liftvti and home_liftvti02 against OCI DRG. You can see them easily by executing the Linux command ifconfig.

• Now all we need to do is to add a Linux OS Route toward those VTI interfaces, to ensure that traffic reaches OCI through the two tunnels. Keep in mind that any traffic directed to a VTI interface will be encapsulated in its associated IPSec tunnel and routed to OCI.

  • The command to add the route has the following format: ip route add {vcnCidrBlock} nexthop dev {vti1} weight {priority}nexthop dev {vti2} weight {priority}`

  • In our test case, we want to be able to reach OCI VCN1 with CIDR 10.0.0.0/16: ip route add 10.0.0.0/16 nexthop dev vti01 weight 1 nexthop dev vti02 weight 1

• Now the ping should work right away.

  ping 10.0.0.109
     PING 10.0.0.109 (10.0.0.109) 56(84) bytes of data.
     64 bytes from 10.0.0.109: icmp_seq=1 ttl=61 time=29.0 ms
     64 bytes from 10.0.0.109: icmp_seq=2 ttl=61 time=29.4 ms
     64 bytes from 10.0.0.109: icmp_seq=3 ttl=61 time=29.0 ms
  

• Now we have two tunnels working.

Task 4: Configure ECMP load balancing and redundancy

This part of the tutorial will focus on ECMP load balancing and redundancy.

• At this point of the tutorial, we currently have two IPSec tunnels established and operational with OCI. Both tunnels are utilizing the identical route towards our on-premises network, as specified in our test case 192.168.0.0/16.

  • Tunnel 1 192.168.0.0/16 –> On-premises

    

  • Tunnel 2 192.168.0.0/16 –> Onprem

    

• Since both tunnels are using same on-premises route (192.168.0.0/16), this route will be propagated toward the rest of the DRG network attachments so other network elements (VCNs, subnets, and so on) will be able reach on-premises via IPSec. As you can see in the next picture, we will have a conflict on route 192.168.0.0/16 that will be the same for the two IP Sec tunnels in the Autogenerated DRG table for VCN attachments.

  

• With this configuration, we will be using just one tunnel and ignoring the other one. The goal of this tutorial is to be able to use both tunnels at the same time in order to achieve high availability (active/active) and double the performance by adding the two tunnels throughput. This is where ECMP comes into play. In order to activate ECMP, we will need first to enable it at the Autogenerated DRG table for every VCN attachment: Networking, Customer connectivity, Dynamic routing gateways, DRGName, DRG route table details, Edit Button (you need to do this on every DRG attachment table).

  

• You will see that the route conflict in Autogenerated DRG route table for VCN attachments is resolved.

  

• Activate ECMP in Linux and Libreswan. In order to support for Equal-cost multi-path routing (ECMP), Linux introduces hash policy choice using fib_multipath_hash_policy, a new sysctl setting that controls which hash policy to use for multipath routes. When fib_multipath_hash_policy is set to 1, the kernel performs L4 hash, which is a multipath hash for IPv4 packets according to a 5-tuple (source IP, source port, destination IP, destination port, IP protocol type) set of values. When fib_multipath_hash_policy is set to 0 (default), only L3 hash is used (the source and destination IP addresses). From Linux console, as root, execute:

  sysctl -w net.ipv4.fib_multipath_hash_policy=1

• Now, we have ECMP activated in both ends, therefore both tunnels will be used in Active/Active mode, adding a total throughput of two IPSec tunnels. For our tests we have used iperf3 command to inspect traffic and got the following results:

   From Libreswan: iperf3 -c 10.0.0.109  -b 1100Mb  -P  8 -t 300
  
   From OCI Origin1: iperf3 -s
  
   [SUM]   0.00-1.75   sec   249 MBytes  1.20 Gbits/sec  7122             sender
  

Related Links

• Site-to-Site VPN Overview

• Equal-Cost Multi-Path (ECMP) Routing on OCI DRGv2

• Site-to-Site VPN Documentation

• Libreswan VPN software

Acknowledgments

Authors - Luis Catalán Hernández (OCI Cloud Network Specialist and Multi Cloud), Antonio Gamir (OCI Cloud Network Specialist)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Use Oracle Cloud Infrastructure Site-to-Site VPN service in HA mode with ECMP routing from Linux and Libreswan

F84216-01

July 2023

Copyright © 2023, Oracle and/or its affiliates.