Protection Mechanisms

IPsec provides two mechanisms for protecting data:

• Authentication Header (AH)

• Encapsulating Security Payload (ESP)

Both mechanisms have their own Security Association Database (SADB).

Authentication Header

The authentication header provides data authentication, strong integrity, and replay protection to IP datagrams. AH protects the greater part of the IP datagram. AH cannot protect fields that change nondeterministically between sender and receiver. For example, the IP TTL field is not a predictable field and, consequently, not protected by AH. AH is inserted between the IP header and the transport header. The transport header can be TCP, UDP, ICMP, or another IP header when tunnels are being used. See the tun(7M) man page for details on tunneling.

Authentication Algorithms and the AH Module

IPsec implements AH as a module that is automatically pushed on top of IP. The /dev/ipsecah entry tunes AH with the ndd command. Future authentication algorithms can be loaded on top of AH. Current authentication algorithms include HMAC-MD5 and HMAC-SHA-1. Each authentication algorithm has its own key size and key format properties. See the authmd5h(7M) and authsha1(7M) man pages for details. For tuning IP configuration parameters, see the ndd(1M) man page.

Security Considerations for AH

Replay attacks threaten an AH when an AH does not enable replay protection. An AH does not protect against eavesdropping. Adversaries can still see data that is protected with AH.

Encapsulating Security Payload

The encapsulating security payload (ESP) header provides confidentiality over what the ESP encapsulates, as well as the services that AH provides. However, ESP only provides its protections over the part of the datagram that ESP encapsulates. ESP's authentication services are optional. These services enable you to use ESP and AH together on the same datagram without redundancy. Because ESP uses encryption-enabling technology, ESP must conform to U.S. export control laws.

ESP encapsulates its data, so ESP only protects the data that follows its beginning in the datagram. In a TCP packet, ESP encapsulates only the TCP header and its data. If the packet is an IP-in-IP datagram, ESP protects the inner IP datagram. Per-socket policy allows self-encapsulation, so ESP can encapsulate IP options when ESP needs to. Unlike the authentication header (AH), ESP allows multiple kinds of datagram protection. Using only a single form of datagram protection can make the datagram vulnerable. For example, if you use ESP to provide confidentiality only, the datagram is still vulnerable to replay attacks and cut-and-paste attacks. Similarly, if ESP protects only integrity, ESP could provide weaker protection than AH. The datagram would be vulnerable to eavesdropping.

Algorithms and the ESP Module

IPsec ESP implements ESP as a module that is automatically pushed on top of IP. The /dev/ipsecesp entry tunes ESP with the ndd command. ESP allows encryption algorithms to be pushed on top of ESP, in addition to the authentication algorithms that are used in AH. Encryption algorithms include Data Encryption Standard (DES), Triple-DES (3DES), Blowfish, and AES. Each encryption algorithm has its own key size and key format properties. Because of export laws in the United States and import laws in other countries, not all encryption algorithms are available outside of the United States. For tuning IP configuration parameters, see the ndd(1M) man page.

Security Considerations for ESP

An ESP without authentication is vulnerable to cut-and-paste cryptographic attacks and to replay attacks. When you use ESP without confidentiality, ESP is as vulnerable to eavesdropping as AH is.

Authentication and Encryption Algorithms

IPsec uses two types of algorithms, authentication and encryption. The authentication algorithms and the DES encryption algorithms are part of core Solaris installation. If you plan to use other algorithms that are supported for IPsec, you must install the Solaris Encryption Kit. The Solaris Encryption Kit is provided on a separate CD.

Authentication Algorithms

Authentication algorithms produce an integrity checksum value or digest that is based on the data and a key. The man pages for authentication algorithms describe the size of both the digest and key. The following table lists the authentication algorithms that are supported in the Solaris operating environment. The table also lists the format of the algorithms when the algorithms are used as security options to the IPsec utilities and their man page names.

Table 1–1 Supported Authentication Algorithms

Algorithm Name	Security Option Format	Man Page
HMAC-MD5	md5, hmac-md5	authmd5h(7M)
HMAC-SHA-1	sha, sha1, hmac-sha, hmac-sha1	authsha1(7M)

Encryption Algorithms

Encryption algorithms encrypt data with a key. The algorithms operate on data in units of a block size. The man pages for encryption algorithms describe the block size and the key size for each algorithm. By default, the DES–CBC and 3DES-CBC algorithms are installed.

The AES and Blowfish algorithms are available to IPsec when you install the Solaris Encryption Kit. The kit is available on a separate CD that is not part of the Solaris 9 installation box. The Solaris 9 Encryption Kit Installation Guide describes how to install the Solaris Encryption Kit.

The following table lists the encryption algorithms that are supported in the Solaris operating environment. The table lists the format of the algorithms when the algorithms are used as security options to the IPsec utilities. The table also lists their man page names, and lists the package that contains the algorithm.

Table 1–2 Supported Encryption Algorithms

Algorithm Name	Security Option Format	Man Page	Package
DES-CBC	des, des-cbc	encrdes(7M)	SUNWcsr, SUNWcarx.u
3DES–CBC or Triple-DES	3des, 3des-cbc	encr3des(7M)	SUNWcsr, SUNWcarx.u
Blowfish	blowfish, blowfish-cbc	encrbfsh(7M)	SUNWcryr, SUNWcryrx
AES-CBC	aes, aes-cbc	encraes(7M)	SUNWcryr, SUNWcryrx