On a FIPS 140-2 enabled system, you are responsible for choosing only FIPS 140-2 approved algorithms when creating certificates and configuring IPsec. The procedures and examples in this guide use FIPS 140-2 approved algorithms except when the algorithm "any" is specified.
The following mechanisms are available to IPsec and approved for use in Oracle Solaris in FIPS 140-2 mode:
AES in CBC, CCM, and GCM modes in 128-bit to 256-bit key lengths
SHA2 in 256-bit to 512-bit key lengths
For the definitive list of FIPS 140-2 approved algorithms for Oracle Solaris, follow the links in FIPS 140-2 Level 1 Certificate References for Oracle Solaris Systems in Using a FIPS 140-2 Enabled System in Oracle Solaris 11.3.