Go to main content

man pages section 4: Device and Network Interfaces

Exit Print View

Updated: Wednesday, July 27, 2022

ipsec (4P)


ipsec - Internet Protocol Security Architecture


The IP Security Architecture (IPsec) provides protection for IP datagrams. The protection can include confidentiality, strong integrity of the data, partial sequence integrity (replay protection), and data authentication. IPsec is performed inside the IP processing, and it can be applied with or without the knowledge of an Internet application.

IPsec applies to both IPv4 and IPv6. See ip(4P) and ip6(4P).

Protection Mechanisms

IPsec provides two mechanisms for protecting data. The Authentication Header (AH) provides strong integrity, replay protection, and data authentication. AH protects as much of the IP datagram as it can. AH cannot protect fields that change non-deterministically between sender and receiver.

The Encapsulating Security Payload (ESP) provides confidentiality over what it encapsulates, as well as the services that AH provides, but only over that which it encapsulates. ESP's authentication services are optional, which allow ESP and AH to be used together on the same datagram without redundancy.

Authentication and encryption algorithms are used for IPsec. Authentication algorithms produce an integrity checksum value or digest-based on the data and a key. Encryption algorithms operate on data in units of a “block size”.

NAT Traversal

IPsec's ESP can also encapsulate itself in UDP if IKE (see in.iked(8)) discovers a Network Address Translator (NAT) between two communicating endpoints.

A UDP socket can be specified as a NAT-Traversal endpoint. See udp(4P) for details.

Security Associations

AH and ESP use Security Associations (SA). SA's are entities that specify security properties from one host to another. Two communicating machines require two SAs (at a minimum) to communicate securely. However, communicating machines that use multicast can share the same multicast SA. SAs are managed through the pf_key(4P) interface. For IPv4, automatic SA management is available through the Internet Key Exchange (IKE), as implemented by in.iked(8). A command-line front-end is available by means of ipseckey(8). An IPsec SA is identified by a tuple of <AH or ESP, destination IP address, and SPI>. The Security Parameters Index (SPI) is an arbitrary 32-bit value that is transmitted on the wire with an AH or ESP packet. See ipsecah(4P) or ipsecesp(4P) for an explanation about where the SPI falls in a protected packet.

Protection Policy and Enforcement Mechanisms

Mechanism and policy are separate. The policy for applying IPsec is enforced on a system-wide or per-socket level. Configuring system-wide policy and per-tunnel policy (see Transport Mode and Tunnel Mode sections) is done via the ipsecconf(8) command. Configuring per-socket policy is discussed later in this section.

System-wide IPsec policy is applied to incoming and outgoing datagrams. Some additional rules can be applied to outgoing datagrams because of the additional data known by the system. Inbound datagrams can be accepted or dropped. The decision to drop or accept an inbound datagram is based on several criteria which sometimes overlap or conflict. Conflict resolution is resolved by which rule is parsed first, with one exception: if a policy entry states that traffic should bypass all other policy, it is automatically be accepted. Outbound datagrams are sent with or without protection. Protection can (or cannot) indicate specific algorithms. If policy normally would protect a datagram, it can be bypassed either by an exception in system-wide policy or by requesting a bypass in per-socket policy.

Intra-machine traffic policies are enforced, but actual security mechanisms are not applied. Instead, the outbound policy on an intra-machine packet translates into an inbound packet with those mechanisms applied.

IPsec policy is enforced in the ip(4P) driver. Several ipadm(8) tunables for IP affect policy enforcement, including:


If equal to on (the default), allow certain cleartext icmp messages to bypass policy. For ICMP echo requests (ping messages), protect the response like the request. If off, treat icmp messages like other IP traffic.


If on, allow inbound cleartext IGMP messages to bypass IPsec policy.


If on, allow inbound cleartext PIM messages to bypass IPsec policy.

Transport Mode and Tunnel Mode

If IPsec is used on a tunnel, Tunnel Mode IPsec can be used to protect distinct flows within a tunnel or to cause packets that do not match per-tunnel policy to drop. System-wide policy is always Transport Mode. A tunnel can use Transport Mode IPsec or Tunnel Mode IPsec.

Per-Socket Policy

The IP_SEC_OPT or IPV6_SEC_OPT socket option is used to set per-socket IPsec policy. The structure used for an IP_SEC_OPT request is:

typedef struct ipsec_req {
    uint_t      ipsr_ah_req;           /* AH request */
    uint_t      ipsr_esp_req;          /* ESP request */
    uint_t      ipsr_self_encap_req;   /* Self-Encap request */
    uint8_t     ipsr_auth_alg;         /* Auth algs for AH */
    uint8_t     ipsr_esp_alg;          /* Encr algs for ESP */
    uint8_t     ipsr_esp_auth_alg;     /* Auth algs for ESP */
} ipsec_req_t;

The IPsec request has fields for both AH and ESP. Algorithms can or cannot be specified. The actual request for AH or ESP services can take one of the following values:


Bypass all policy. Only a user granted with the PRIV_SYS_IP_CONFIG privilege can request this service.


Regardless of other policy, require the use of the IPsec service.

The following value can be logically ORed to an IPSEC_PREF_REQUIRED value:


Regardless of other policy, enforce a unique SA for traffic originating from this socket.

In the event IP options not normally encapsulated by ESP need to be, the ipsec_self_encap_req is used to add an additional IP header outside the original one. Algorithm values from <net/pfkeyv2.h> are as follows:


Uses the Camellia algorithm for encryption.


Uses the SHA2 hash algorithms with HMAC (RFC 4868) for authentication.

An application should use either the getsockopt(3C) or the setsockopt(3C) call to manipulate IPsec requests. For example:

#include <sys/socket.h>
#include <netinet/in.h>
#include <net/pfkeyv2.h>   /* For SADB_*ALG_* */
/* .... socket setup skipped */
rc = setsockopt(s, IPPROTO_IP, IP_SEC_OPT,
   (const char *)&ipsec_req, sizeof (ipsec_req_t)); 


While IPsec is an effective tool in securing network traffic, it does not make security problems disappear. Security issues beyond the mechanisms that IPsec offers can be discussed in similar ”Security” or “Security Consideration” sections within individual reference manual pages.

While an unprivileged user can never bypass IPsec, it is possible to allow an unprivileged user to set per-socket policy to be different from the system-wide policy.

The following ipadm tunable for IP controls this behavior:


If equal to on (the default), require the PRIV_SYS_IP_CONFIG privilege in order to set the algorithms in per-socket policy different from the system-wide policy. If equal to off, allow an unprivileged user to change the algorithms, but not to bypass policy altogether.


IPsec policy logging is managed by the service management facility, smf(7), under the service identifier:


The policy logger uses dtrace(8) to retrieve details of IPsec policy failures and other types of errors and record them in a log file. The administrator can use this information to determine the cause of IPsec failures. This service instance can only be enabled from a global zone.

Administrative actions on this service instance, such as enabling, disabling, or requesting restart, can be performed using svcadm(8). The service instance status can be queried using the svcs(1) command. This service instance is delivered disabled, with the default log level set to message+packet. In order to use this service instance, you must be superuser or be granted the Network IPsec Management rights profile.

Refresh rereads properties associated with this service instance. Log file rotation is optionally performed by the start method.


The section below describes a number of smf(7) properties that are used by the policy logger. This service instance needs to be refreshed and restarted using svcadm(8) before a new property value is effective. For example:

# svcadm refresh policy:logger
# svcadm restart policy:logger

    The following properties are defined for the logger instance of the policy service:

  • config/log_level

  • config/log_file

  • config/log_rotation_size

  • config/log_size_units

  • config/backtrace_depth

Each of these properties are described in detail in the following sections.


Defines which of the following are logged when the dtrace probes fire: messages, stack back trace, or partial packet decode. The default value is message+packet. Stack back trace and partial packet decoding are captured immediately before the packet is dropped.

The log_level property is a string consisting of one or more descriptive substrings. Each substring enables a different type of information logging. The valid substrings are minimal, message, packet, and stack, combined in any order. If the substring minimal is present in the string, all other types of logging are excluded. The following are valid examples for log_level:

log messages and partial packet decode
log messages, stack back trace, and partial packet decode
log partial packet decode only

The following command changes the value of the log_level property to log all possible information. This must be followed by a refresh to update SMF. The service instance must be restarted for the new property to take effect.

# svccfg -s ipsec/policy:logger setprop config/log_level = \

Defines where log output should be written. The default is /var/log/ipsec/ipsec.log. The logger appends information to the active log file upon restart if no log rotation is performed. It is important that log_file is writable by the userid daemon or the service instance will fail to start.

Use the following command to examine the log_file property:

# svcprop policy:logger | grep log_file

To specify a new log file called new_file, contained in directory new_dir, use the following command:

# svccfg -s policy:logger setprop config/log_file= \

Startup error messages are recorded by the smf(7) framework and recorded in a service-specific log file.


Along with the log_size_units property, defines the size threshold for log file rotation. As long as the log file size is less than this value, service instance restarts will not perform log file rotation.

The following command changes the value of the log_rotation_size property to 25.

# svccfg -s ipsec/policy:logger setprop \
config/log_rotation_size = 25

Defines the units for the log_rotation_size. This is a single upper or lowercase character. Valid values are the letter b for bytes, k for kbytes, m for mbytes or g for gbytes.

The following command changes the value of the log_size_units property to indicate megabytes.

# svccfg -s ipsec/policy:logger setprop config/log_size_units = m

The default value for log_rotation_size is 1, and the default value for log_size_units is b, indicating 1 byte, for rotate on demand behavior. To accumulate logging results into a single file, across service instance restarts, the log_rotation_size or log_size_units properties can be set to a custom value.


Defines the stack back trace depth from the kernel location where the packet is dropped, in number of entries. Ignored if the log_level property does not have stack back trace logging enabled. The default depth is 25. The following example changes the stack back trace depth to 6 entries.

# svccfg -s ipsec/policy:logger setprop config/backtrace_depth=6


Example 1 Disabling the Service Instance

The following command disables the service instance.

# svcadm disable svc:/network/ipsec/policy:logger
Example 2 Enabling the Service Instance

The following command enables the service instance. If the log file size is greater than or equal to the configured threshold value, log file rotation is performed before logging is started.

Log rotation renames the corresponding log file by adding a suffix so that the most recent old log file ends with .0 (that is, log_file.0), the next most recent ends with .1 (that is, log_file.1), and so forth. Ten versions of old log files are preserved (that is, log_file.0 through log_file.9). At the next rotation after log_file.9 is created, the oldest version is deleted to keep the count of files at 10 old log files and one active log file.

# svcadm enable svc:/network/ipsec/policy:logger
Example 3 Refreshing the Service Instance

The following command refreshes the service instance.

# svcadm refresh svc:/network/ipsec/policy:logger
Example 4 Restarting the Service Instance

The following command restarts the service instance.

# svcadm restart svc:/network/ipsec/policy:logger



Default log file.


See attributes(7) for descriptions of the following attributes:

Interface Stability

See Also

getsockopt(3C), setsockopt(3C), inet(4P), ip(4P), ip6(4P), ipsecah(4P), ipsecesp(4P), pf_key(4P), udp(4P), attributes(7), in.iked(8), ipadm(8), ipsecconf(8), ipseckey(8), ndd(8)

Kent, S., and Atkinson, R., RFC 2401, Security Architecture for the Internet Protocol, The Internet Society, 1998.


Kent, S. and Atkinson, R., RFC 2406, IP Encapsulating Security Payload (ESP), The Internet Society, 1998.


Madson, C., and Doraswamy, N., RFC 2405, The ESP DES-CBC Cipher Algorithm with Explicit IV, The Internet Society, 1998.


Madsen, C. and Glenn, R., RFC 2403, The Use of HMAC-MD5-96 within ESP and AH, The Internet Society, 1998.


Madsen, C. and Glenn, R., RFC 2404, The Use of HMAC-SHA-1-96 within ESP and AH, The Internet Society, 1998.


Pereira, R. and Adams, R., RFC 2451, The ESP CBC-Mode Cipher Algorithms, The Internet Society, 1998.


Kelly, S. and Frankel, S., RFC 4868, Using HMAC-SHA-256, HMAC-SHA-384, and HMAC-SHA-512 with IPsec, The Internet Society, 2007.


Huttunen, A., Swander, B., Volpe, V., DiBurro, L., Stenberg, M., RFC 3948, UDP Encapsulation of IPsec ESP Packets, The Internet Society, 2005.