How to Set Up a Virtual Private Network (VPN) (IPsec and IKE Administration Guide)

IPsec and IKE Administration Guide

How to Set Up a Virtual Private Network (VPN)

This procedure shows you how to set up a VPN by using the Internet to connect two networks within an organization. The procedure then shows you how to secure the traffic between the networks with IPsec.

This procedure extends the procedure, How to Secure Traffic Between Two Systems. In addition to connecting two systems, you are connecting two intranets that connect to these two systems. The systems in this procedure function as gateways.

This procedure assumes the following setup:

Each system is using an IPv4 address space. This procedure also works with IPv6 addresses.
Each system has two interfaces. The hme0 interface connects to the Internet. In this example, Internet IP addresses begin with 192.168. The hme1 interface connects to the company's local area network (LAN), its intranet. In this example, intranet IP addresses begin with the number 10.
Each system invokes AH protection with the MD5 algorithm. The MD5 algorithm requires a 128-bit key.
Each system invokes ESP protection with the 3DES algorithm. The 3DES algorithm requires a 192-bit key.
Each system can connect to a router that has direct access to the Internet.
IPsec uses shared security associations (SAs).

For a description of VPNs, see Virtual Private Networks. The following figure describes the VPN that this procedure configures.

Diagram shows details of VPN between Europe and California offices.

This procedure uses the following configuration parameters.

Parameter	Europe	California
System name	enigma	partym
System intranet interface	hme1	hme1
System Internet interface	hme0	hme0
System intranet address, also the `-point` address in Step 8	10.16.16.6	10.1.3.3
System Internet address, also the `-taddr` address in Step 8	192.168.116.16	192.168.13.213
Name of Internet router	router-E	router-C
Address of Internet router	192.168.116.4	192.168.13.5
Tunnel name	`ip.tun0`	`ip.tun0`

On the system console on one of the systems, become superuser or assume an equivalent role.

Note –
Logging in remotely exposes security-critical traffic to eavesdropping. Even if you somehow protect the remote login, the security of the system is reduced to the security of the remote login session.

Turn off IP forwarding. Choose one of the following options:
- On IPv4 networks, use this syntax:
  # ndd -set /dev/ip ip_forwarding 0
- On IPv6 networks, use this syntax:
  # ndd -set /dev/ip6 ip6_forwarding 0
Turning off IP forwarding prevents packets from being forwarded from one network to another network through this system. For a description of the ndd command, see the ndd(1M) man page.

Turn on IP strict destination multihoming. Choose one of the following options:
- On IPv4 networks, use this syntax:
  # ndd -set /dev/ip ip_strict_dst_multihoming 1
- On IPv6 networks, use this syntax:
  # ndd -set /dev/ip6 ip6_strict_dst_multihoming 1
Turning on IP strict destination multihoming ensures that packets for one of the system's destination addresses arrive at the correct destination address.

When you use the ndd command to turn off IP forwarding and turn on IP strict destination multihoming, fewer packets flow all the way through the system. When strict destination multihoming is enabled, packets that arrive on a particular interface must be addressed to one of the local IP addresses of that interface. All other packets, even ones addressed to other local addresses of the system, are dropped.

Disable most network services, and possibly all network services, by doing the following substeps, as needed:
1. Edit the inetd.conf file to remove all but essential services. Then, force the inetd daemon to reread the inetd.conf file.
  # pkill -HUP inetd
  Note –
  The VPN router should allow very few incoming requests. You need to disable all processes that accept incoming traffic. For example, you might comment out lines in the inetd.conf file, you might kill SNMP, and so on. Alternatively, you can use techniques that are similar to the techniques in How to Secure a Web Server.
2. If the inetd.conf file has not been edited to remove all but essential services, kill the the inetd daemon.
  # pkill inetd
3. Disable other Internet services, such as SNMP, NFS, and so on, by typing the appropriate commands. For example, the following commands kill the NFS service and the mail service:
  # /etc/init.d/nfs.server stop # /etc/init.d/sendmail stop
  The disabling of network services prevents IP packets from doing any harm to the system. For example, an SNMP daemon, a telnet connection, or an rlogin connection could be exploited.

On each system, add a pair of SAs between the two systems.

Configure IKE to manage the keys for the SAs. Use one of the procedures in Configuring IKE (Task Map) to configure IKE for the VPN.

If you have an overriding reason to manually manage the keys, see How to Manually Create IPsec Security Associations.

On each system, edit the /etc/inet/ipsecinit.conf file to add the VPN policy.
1. For example, on the enigma system, type the following entries into the ipsecinit.conf file:
  # LAN traffic can bypass IPsec. {laddr 10.16.16.6 dir both} bypass {} # WAN traffic uses ESP with 3DES and MD5. {} ipsec {encr_algs 3des encr_auth_algs md5}
2. On the partym system, type the following entries into the ipsecinit.conf file:
  # LAN traffic can bypass IPsec. {laddr 10.1.3.3 dir both} bypass {} # WAN traffic uses ESP with 3DES and MD5. {} ipsec {encr_algs 3des encr_auth_algs md5}
The ipsec entry prevents remote systems from sending clear packets. The bypass entry allows nodes that are part of the LAN to treat the VPN router as if the router were part of the LAN.

(Optional) For a higher level of security, remove the LAN bypass entry.

The entry in the ipsecinit.conf file would appear similar to the following:
# All traffic uses ESP with 3DES and MD5. {} ipsec {encr_algs 3des encr_auth_algs md5}
Each system on the LAN would then need to activate IPsec to communicate with the VPN router.

On each system, configure a secure tunnel, ip.tun0.

The tunnel adds another physical interface from the IP perspective. Type the following three ifconfig commands to create the point-to-point interface:

# ifconfig ip.tun0 plumb

# ifconfig ip.tun0 system1-point system2-point \
tsrc system1-taddr tdst system2-taddr encr_algs 3DES encr_auth_algs MD5

# ifconfig ip.tun0 up

For example, on the enigma system, type the following commands:

# ifconfig ip.tun0 plumb

# ifconfig ip.tun0 10.16.16.6 10.1.3.3   \
tsrc 192.168.116.16 tdst 192.168.13.213 \
encr_algs 3DES encr_auth_algs MD5

# ifconfig ip.tun0 up

On the partym system, type the following commands:

# ifconfig ip.tun0 plumb

# ifconfig ip.tun0 10.1.3.3 10.16.16.6  \
tsrc 192.168.13.213 tdst 192.168.116.16 \
encr_algs 3DES encr_auth_algs MD5

# ifconfig ip.tun0 up

The policy that is passed to the ifconfig commands must be the same as the policy in the ipsecinit.conf file. Upon reboot, each system uses the policy in its ipsecinit.conf file.

On each system, turn on IP forwarding for the hme1 and ip.tun0 interfaces. Choose one of the following options:
- On IPv4 networks, use this syntax:
  # ndd -set /dev/ip hme1:ip_forwarding 1 # ndd -set /dev/ip ip.tun0:ip_forwarding 1
- On IPv6 networks, use this syntax:
  # ndd -set /dev/ip6 hme1:ip6_forwarding 1 # ndd -set /dev/ip6 ip.tun0:ip6_forwarding 1
IP forwarding means that packets that arrive from somewhere else can be forwarded. IP forwarding also means that packets that leave this interface might have originated somewhere else. To successfully forward a packet, both the receiving interface and the transmitting interface must have IP forwarding turned on.

Because the hme1 interface is inside the intranet, IP forwarding must be turned on for hme1. Because ip.tun0 connects the two systems through the Internet, IP forwarding must be turned on for ip.tun0.

The hme0 interface has its IP forwarding turned off to prevent an outside adversary from injecting packets into the protected intranet. The outside refers to the Internet.

On each system, ensure that routing protocols do not advertise the default route within the intranet.
# ifconfig hme0 private
Even if hme0 has IP forwarding turned off, a routing protocol implementation might still advertise the interface. For example, the in.routed protocol might still advertise that hme0 is available to forward packets to its peers inside the intranet. Setting the interface's private flag prevents these advertisements.

Manually, add a default route over hme0.

The default route should be a router with direct access to the Internet.
# pkill in.rdisc # route add default router-on-hme0-subnet
1. For example, on the enigma system, add the following route:
  # pkill in.rdisc # route add default 192.168.116.4
2. On the partym system, add the following route:
  # pkill in.rdisc # route add default 192.168.13.5
  Even though the hme0 interface is not part of the intranet, hme0 does need to reach across the Internet to its peer system. To find its peer, hme0 needs information about Internet routing. The VPN system appears to be a host, rather than a router, to the rest of the Internet. Therefore, you can use a default router or run the router discovery protocol to find a peer system. For more information, see the route(1M) and in.routed(1M) man pages.

Ensure that hme0 uses the default route after a reboot by creating a defaultrouter file.

Put the IP address of hme0's default router in the /etc/defaultrouter file. This step prevents the in.rdisc daemon from being started at reboot.
1. For example, on the enigma system, add the Internet router for enigma in the /etc/defaultrouter file.
  # vi /etc/defaultrouter 192.168.116.4 router-E
2. Add the partym system's Internet router in partym's /etc/defaultrouter file.
  # vi /etc/defaultrouter 192.168.13.5 router-C

On each system, prevent routing from occurring early in the boot sequence, thus reducing vulnerability.
# touch /etc/notrouter

Ensure that the VPN starts after a reboot by editing the /etc/hostname.ip.tun0 file.

system1-point system2-point tsrc system1-taddr \
tdst system2-taddr encr_algs 3des encr_auth_algs md5 up

For example, on the enigma system, add the following lines to the hostname.ip.tun0 file:

10.16.16.6 10.1.3.3 tsrc 192.168.116.16 \
tdst 192.168.13.213 encr_algs 3DES encr_auth_algs MD5 up

On the partym system, add the following lines to the hostname.ip.tun0 file:

10.1.3.3 10.16.16.6 tsrc 192.168.13.213 \
tdst 192.168.116.16 encr_algs 3DES encr_auth_algs MD5 up

On each system, create a file that configures some VPN parameters at boot time. Name the file /etc/rc3.d/S99vpn_setup.

On each system, turn on IP forwarding for the hme1 and ip.tun0 interfaces. Choose one of the following options:
- On an IPv4 network, type the following lines in the file:
  ndd -set /dev/ip hme1:ip_forwarding 1 ndd -set /dev/ip ip.tun0:ip_forwarding 1 ifconfig hme0 private in.routed
- On an IPv6 network, type the following lines in the file:
  ndd -set /dev/ip6 hme1:ip6_forwarding 1 ndd -set /dev/ip6 ip.tun0:ip6_forwarding 1 ifconfig hme0 private in.routed
You can also manually add routes in the /etc/rc3.d/S99vpn_setup file, instead of using the in.routed protocol.

On each system, run a routing protocol.
# in.routed