Protection Mechanisms

IPsec provides two mechanisms for protecting data:

• Authentication Header (AH)

• Encapsulating Security Payload (ESP)

Both mechanisms use security associations.

Authentication Header

The authentication header, a new IP header, provides strong integrity, partial sequence integrity (replay protection), and data authentication to IP datagrams. AH protects as much of the IP datagram as it can. AH cannot protect fields that change nondeterministically between sender and receiver. For example, the IP TTL field is not a predictable field and, consequently, not protected by AH. AH is inserted between the IP header and the transport header. The transport header can be TCP, UDP, ICMP, or another IP header when tunnels are being used. See the tun(7M) man page for details on tunneling.

Authentication Algorithms and the AH Module

IPsec implements AH as a module that is automatically pushed on top of IP. The /dev/ipsecah entry tunes AH with ndd(1M). Future authentication algorithms can be loaded on top of AH. Current authentication algorithms include HMAC-MD5 and HMAC-SHA-1. Each authentication algorithm has its own key size and key format properties. See the authmd5h(7M) and authsha1(7M) man pages for details.

Security Considerations

Replay attacks threaten any AH that does not enable replay protection. An AH does not protect against eavesdropping. Adversaries can still see data protected with AH.

Encapsulating Security Payload

The ESP provides confidentiality over what it encapsulates, as well as the services that AH provides, but only over that which it encapsulates. ESP's authentication services are optional. These services enable you to use ESP and AH together on the same datagram without redundancy. Because ESP uses encryption-enabling technology, it must conform to U.S. export control laws.

ESP encapsulates its data, so it only protects the data that follows its beginning in the datagram. In a TCP packet, ESP encapsulates only the TCP header and its data. If the packet is an IP-in-IP datagram, ESP protects the inner IP datagram. Per-socket policy allows self-encapsulation, so ESP can encapsulate IP options when it needs to. Unlike the authentication header (AH), ESP allows multiple kinds of datagram protection. Using only a single form of datagram protection can make the datagram vulnerable. For example, if you use ESP to provide confidentiality only, the datagram is still vulnerable to replay attacks and cut-and-paste attacks. Similarly, if ESP protects only integrity and does not fully protect against eavesdropping, it could provide weaker protection than AH.

Algorithms and the ESP Module

IPsec ESP implements ESP as a module that is automatically pushed on top of IP. The /dev/ipsecesp entry tunes ESP with ndd(1M). ESP allows encryption algorithms to be pushed on top of it, in addition to the authentication algorithms that are used in AH. Encryption algorithms include United States Data Encryption Standard (DES), Triple-DES (3DES), Blowfish, and AES. Each encryption algorithm has its own key size and key format properties. Because of export laws in the United States and import laws in other countries, not all encryption algorithms are available outside of the United States.

Security Considerations

An ESP without authentication is vulnerable to cut-and-paste cryptographic attacks and to eavesdropping attacks. When you use ESP without confidentiality, ESP is as vulnerable to replay as AH is.

Authentication and Encryption Algorithms

IPsec uses two types of algorithms, authentication and encryption. The authentication algorithms and the DES encryption algorithms are part of core Solaris installation. If you plan to use other algorithms that are supported for IPsec, you must install the Solaris Encryption Kit, which is provided on a separate CD.

Authentication Algorithms

Authentication algorithms produce an integrity checksum value or digest that is based on the data and a key. The authentication algorithm man pages describe the size of both the digest and key. The following table lists the authentication algorithms that are supported in the Solaris operating environment. The table also lists the format of the algorithms when they are used as security options to the IPsec utilities and their man page names.

Table 19–1 Supported Authentication Algorithms

Algorithm Name	Security Option Format	Man Page
HMAC-MD5	md5, hmac-md5	authmd5h(7M)
HMAC-SHA-1	sha, sha1, hmac-sha, hmac-sha1	authsha1(7M)

Encryption Algorithms

Encryption algorithms encrypt data with a key. The algorithms operate on data in units of a block size. The encryption algorithm man pages describe the size of both the block size and the key size. By default, the DES-CBC and 3DES-CBC algorithms are installed. You must install the Solaris Encryption Kit to make the AES and Blowfish algorithms available to IPsec. The kit is available on a separate CD that is not part of the Solaris 9 installation box. The Encryption Kit Installation Guide describes how to install the Solaris Encryption Kit.

The following table lists the encryption algorithms that are supported in the Solaris operating environment. The table also lists the format of the algorithms when they are used as security options to the IPsec utilities, their man page names, and the package that contains them.

Table 19–2 Supported Encryption Algorithms

Algorithm Name	Security Option Format	Man Page	Package
DES-CBC	des, des-cbc	encrdes(7M)	SUNWcsr, SUNWcarx.u
3DES–CBC or Triple-DES	3des, 3des-cbc	encr3des(7M)	SUNWcsr, SUNWcarx.u
Blowfish	blowfish, blowfish-cbc	encrbfsh(7M)	SUNWcryr, SUNWcryrx
AES-CBC	aes, aes-cbc	encraes(7M)	SUNWcryr, SUNWcryrx