IPsec policy can be applied at the per-socket level and the system-wide level.
IPsec applies policy to outbound packets and inbound packets that match an IPsec policy rule. Each policy rule can have one or more actions. An action could be to encrypt a packet by using a specific algorithm, or to pass the packet without encryption. To specify multiple acceptable algorithms, a policy rule would use multiple actions.
An IPsec policy rule has three parts.
Selectors – Determine if a rule matches the network packet. Also known as traffic selectors. An empty ({}) selector matches all traffic. A selector can have more than one action/parameter pair.
Action – Applied when traffic matches the selectors. Examples of actions include ipsec and pass.
Action parameters – Additional specifications for an action. Simple actions like pass or drop do not have parameters. Actions such as ipsec can specify cryptographic parameters.
IPsec policy is applied to both inbound and outbound packets. A packet that does not match any rule is passed. When packets can match more than one rule, the first match is used.
The rules are processed in the following order:
Per-socket rules
System-wide pass, bypass, and drop rules
System-wide ipsec rules that use ESP
System-wide ipsec rules that use AH
The bypass and or pass options specify exceptions to an IPsec policy rule that otherwise applies to the packets.
You can bypass all or part of an IPsec rule. Packets matching a bypass rule will be allowed to pass without IPsec protection and any other IPsec policy rules that match the packets are not applied. For example, packets from web clients might not need to be encrypted. See How to Use IPsec to Protect Web Server Communication With Other Servers.
The or pass action in an IPsec policy rule enables non-IPsec packets that match a previous action in the rule to pass into the system. An IPsec policy rule which has an encrypt action and an or pass action accepts encrypted packets and packets that are not encrypted from the client systems.
The or pass action enables a server to serve clients that are not configured with IPsec as well as clients that are. One example of use would be when a network is in transition to configuring IPsec on every system. This option is not suitable for an environment where all traffic must be encrypted. For an example, see Example 34, Transitioning Client Systems to Use IPsec by Using the or pass Action on the Server.
You use the ipsecinit.conf file and the ipsecconf command to configure IPsec policy. For details and examples, see the ipsecconf(1M) man page and Configuring IPsec.