Unable to Access SMB Shares from Windows 10 Client When TryIPSPN Registry Is Enabled

Starting with Windows 10, clients can enable Kerberos authentication with IP address-based Service Principal Names (SPNs) by creating a TryIPSPN registry entry. However, Oracle Solaris does not support IP address-based SPNs.

Like Windows, Oracle Solaris does not register IP address-based SPNs as part of the AD domain join. Microsoft notes the following potential issues with IP address-based SPNs: IP addresses are not normally used in place of hostnames because IP addresses are often temporary. Using IP addresses can lead to conflicts and authentication failures as address leases expire and renew. Therefore, registering an IP address-based SPN is a manual process and should only be used when it is not possible to switch to a DNS-based hostname.

When the TryIPSPN registry setting is enabled, Windows 10 clients can continue to access SMB shares exported by the Oracle Solaris system via NTLMSSP. If the domain administrator has manually added cifs\IP-address SPNs to the ServicePrincipalName attribute of the AD computer object, Windows KDC will be able to issue CIFS service tickets for the Oracle Solaris system using IP address-based SPNs. However, if the client subsequently presents a CIFS service ticket that uses IP address-based SPNs to access SMB resources on the system, the Oracle Solaris Kerberos subsystem will reject the security context due to the lack of support for IP address-based SPNs. As a result, syslog will contain the following notice-level messages:

krbssp: user authentication failed (GSS major error): No credentials were
supplied, or the credentials were unavailable or inaccessible
krbssp: user authentication failed (GSS minor error): No principal in keytab
('FILE:/var/krb5/krb5.keytab') matches desired name cifs/IP-address@AD-realm

To avoid this issue, do not publish IP address-based SPNs to an AD server.