The IPsec standards define two distinct modes of IPsec operation, transport mode and tunnel mode. The key difference between transport and tunnel mode is where policy is applied. In tunnel mode, the original packet is encapsulated in an outer IP header. The IP addresses in the inner and outer headers can be different.
Traffic selectors, introduced in IPsec Policy, determine if a packet matches a policy rule. Selectors include:
Source IP address
Destination IP address
Protocol number, if applicable
Port numbers, if applicable
The pattern used to match IPsec policy rules consists of a subset of these selectors.
In transport mode, the traffic selectors are matched against the outer IP header. In tunnel mode, they are matched against the inner IP header. Tunnel mode can be applied to any mix of end systems and intermediate systems, such as security gateways.
In transport mode, the IP header, the next header, and any ports that the next header supports can be used to determine if IPsec policy applies. In effect, IPsec can enforce different transport mode policies between two IP addresses to the granularity of a single port. For example, if the next header is TCP, which supports ports, then IPsec policy can be set for a TCP port of the outer IP address.
Tunnel mode works only for IP-in-IP packets. In tunnel mode, IPsec policy is enforced on the contents of the inner IP packet. Different policy can be enforced for different inner IP addresses. That is, the inner IP header, its next header, and the ports that the next header supports can enforce a policy.
In tunnel mode, IPsec policy can be specified for subnets of a LAN behind a router and for ports on those subnets. IPsec policy can also be specified for particular IP addresses, that is, hosts, on those subnets. The ports of those hosts can also have a specific IPsec policy. For examples of tunneling procedures that include configuring static routes, see Protecting a VPN With IPsec.
When IPsec policy is applied to traffic in IP tunnels, the name of the IP tunnel interface is used to link the traffic in that tunnel to an IPsec policy rule. IPsec policy provides a tunnel keyword to select an IP tunneling network interface. When the tunnel keyword is present in a rule, all selectors that are specified in that rule apply to the inner packet.
For information about tunneling interfaces, see Chapter 4, About IP Tunnel Administration in Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.4.
The following figures illustrate protected and unprotected packets.
Unprotected IP Packet Carrying TCP Information shows an IP header with an unprotected TCP packet.
Figure 5 Unprotected IP Packet Carrying TCP Information
Protected IP Packet Carrying TCP Information shows ESP protecting the data in transport mode. The shaded area shows the encrypted part of the packet.
Figure 6 Protected IP Packet Carrying TCP Information
IPsec Packet Protected in Tunnel Mode shows that the entire packet is inside the ESP header in tunnel mode. The packet from Unprotected IP Packet Carrying TCP Information is protected in tunnel mode by an outer IPsec header and, in this case, ESP.
Figure 7 IPsec Packet Protected in Tunnel Mode
IPsec policy provides keywords for tunnel mode and transport mode. For more information, review the following:
For details on per-socket policy, see the ipsec(4P) man page.
For an example of per-socket policy, see How to Use IPsec to Protect Web Server Communication With Other Servers.
For more information about tunnels, see the ipsecconf(8) man page.
For an example of tunnel configuration, see How to Protect the Connection Between Two LANs With IPsec in Tunnel Mode.