System Administration Guide, Volume 3

Protection Mechanisms

IPsec provides two mechanisms for protecting data:

Both mechanisms use security associations.

Authentication Header

The authentication header, a new IP header, provides strong integrity, partial sequence integrity (replay protection), and data authentication to IP datagrams. AH protects as much of the IP datagram as it can. AH cannot protect fields that change nondeterministically between sender and receiver. For example, the IP TTL field is not a predictable field and, consequently, not protected by AH. AH is inserted between the IP header and the transport header. The transport header can be TCP, UDP, ICMP, or another IP header when tunnels are being used. See the tun(7M) man page for details on tunneling.

Authentication Algorithms and the AH Device

IPsec implements AH as a module that is automatically pushed on top of IP. The /dev/ipsecah entry tunes AH with ndd(1M), in addition to allowing future authentication algorithms to be loaded on top of AH. Current authentication algorithms include HMAC-MD5 and HMAC-SHA-1. Each authentication algorithm has its own key size and key format properties. See the authmd5h(7M) and authsha1(7M) man pages for details.

Security Considerations

Without replay protection enabled, all replay attacks jeopardize AH. AH does not protect against eavesdropping. Adversaries can still see data protected with AH.

Encapsulating Security Payload

The ESP provides confidentiality over what it encapsulates, as well as the services that AH provides, but only over that which it encapsulates. ESP's authentication services are optional. These services enable you to use ESP and AH together on the same datagram without redundancy. Because ESP uses encryption-enabling technology, it falls under U.S. export control laws.

ESP encapsulates its data, so it only protects the data that follows its beginning in the datagram. In a TCP packet, ESP encapsulates only the TCP header and its data. If the packet is an IP in IP datagram, ESP protects the inner IP datagram. Per-socket policy allows self-encapsulation, so ESP can encapsulate IP options, when it needs to. Unlike the authentication header (AH) , ESP allows multiple kinds of datagram protection. Using only a single form of datagram protection can expose the datagram to vulnerabilities. For example, you can use only ESP to provide confidentiality, but protecting confidentiality alone exposes vulnerabilities in both replay attacks and cut-and-paste attacks. Similarly, if ESP protects only integrity, and does not fully protect against eavesdropping, it could provide weaker protection than AH.

Algorithms and the ESP Device

IPsec ESP implements ESP as a module that is automatically pushed on top of IP. Use the /dev/ipsecesp entry to tune ESP with ndd(1M), as well as to allow future algorithms to be loaded on top of ESP. ESP allows encryption algorithms to be pushed on top of it, in addition to the authentication algorithms used in AH. Encryption algorithms include United States Data Encryption Standard (DES) and Triple-DES (3DES). Each encryption algorithm has its own key size and key format properties. Because of export laws in the United States, not all encryption algorithms are available outside of the United States.

Security Considerations

ESP without authentication exposes vulnerabilities to cut-and-paste cryptographic attacks, as well as eavesdropping attacks. When you use ESP without confidentiality, its vulnerability to replay is similar to AH. Because of United States export control laws, the encryption strength available on ESP is weaker for versions of the SunOS sold outside the United States.

Authentication and Encryption Algorithms

IPsec uses two types of algorithms:

Authentication Algorithms

Authentication algorithms produce an integrity checksum value or digest based on the data and a key. The authentication algorithm man pages describe the size of both the digest and key (see, for example, authmd5h(7M) and authsha1(7M) man pages).

Encryption Algorithms

Encryption algorithms encrypt data with a key. Encryption algorithms operate on data in units of a block size. The encryption algorithm man pages describe the size of both the block size and the key size (see, for example, encrdes(7M) and encr3des(7M) man pages).