Sun OpenSSO Enterprise 8.0 Administration Guide

Configuring the Authentication Process

As discussed in Understanding the Authentication Service the authentication process itself is configured (simply) by instantiating the desired authentication module or (with more complexity) by creating an authentication chain of modules. The following sections contain the procedures for configuring an authentication process. (If you are configuring a process that includes the RADIUS, SafeWord, or Windows-based authentication modules, see Before You Begin.)

Before You Begin

The procedures in this section are relevant when using the RADIUS, SafeWord, or the Windows-based authentication modules.

Setting Up for RADIUS and SafeWord Authentication

To Set Up RADIUS or SafeWord with Sun Java System Application Server should be performed before configuring an authentication process that uses the RADIUS or SafeWord authentication modules.

ProcedureTo Set Up RADIUS or SafeWord with Sun Java System Application Server

A Java Platform, Enterprise Edition SocketPermission class represents access to a network using sockets; it consists of a host location and a set of actions specifying ways to connect to that host. When the SafeWord client forms a socket connection to its server, only the connect action of the SocketPermission object is allowed in the Application Server’s server.policy file. In order for the SafeWord authentication module to work properly, permission needs to be granted to the accept, listen, and resolve actions manually.

  1. Open the server.policy file in a text editor.

  2. Add an entry for the appropriate actions into the Application Server server.policy file.

    For example, permission "localhost:1024-", "accept,connect,listen";

    If this permission is granted to some code, it allows that code to accept connections on, connect to, or listen to any port between 1024 and 65535 on the local host. The listen action is only meaningful when used with a local host. The resolve (resolve host/IP name service lookups) action is implied when any of the other actions are present. This second example (permission "", "connect,accept";) allows the code to connect to, and accept connections on, port 1645 on

    Note –

    Granting code permission to accept or make connections to remote hosts may cause problems, because malevolent code can then more easily transfer and share confidential data among parties who may not otherwise have access to the data. Make sure to give only appropriate permissions by specifying exact port number instead of allowing a range of port numbers

  3. Save the server.policy file.

  4. Restart Application Server.

See Also

For more information on SocketPermission, see the Java Platform, Enterprise Edition API Specification

Setting Up Windows Desktop SSO Authentication

In addition to configuring the Windows Desktop SSO authentication module using the OpenSSO Enterprise console, a user must be created in the Windows 2000 Domain Controller and Internet Explorer must be configured.

ProcedureTo Create a User in the Windows Domain Controller

  1. In the domain controller, create a user account for the OpenSSO Enterprise authentication module.

    1. From the Start menu, go to Programs>Administration Tools.

    2. Select Active Directory Users and Computers.

    3. Go to Computers > New > computer and add the client computer's name.

      If you are using Windows XP, this step is performed automatically during the domain controller account configuration.

    4. Go to Users > New > Users and create a new user with the OpenSSO Enterprise host name as the User ID (login name).

      The OpenSSO Enterprise host name should not include the domain name.

  2. Associate the user account with a service provider name.

  3. Install the ktpass utilities to the c:\program files\support tools directory.

    The ktpass utilities are not installed as part of the Windows 2000 server. You must install it from the installation CD.

  4. Export the keytab files to the system in which OpenSSO Enterprise is installed by running the following commands.

    ktpass -princ host/hostname.domainname@DCDOMAIN -pass password -mapuser userName-out
    ktpass -princ HTTP/hostname.domainname@DCDOMAIN -pass 
    password -mapuser userName-out hostname.HTTP.keytab

    The ktpass command accepts the following parameters:

    hostname. The host name (without the domain name) on which OpenSSO Enterprise runs.

    domainname . The OpenSSO Enterprise domain name.

    DCDOMAIN. The domain name of the domain controller. This may be different from the OpenSSO Enterprise domain name.

    password . The password of the user account. Make sure that password is correct, as ktpass does not verify passwords.

    userName. The user account ID. This should be the same as hostname.

    Note –

    Make sure that both keytab files are kept secure.

    The service template values should be similar to the following example:

    Service Principal: HTTP/machine1.EXAMPLE.COM@ISQA.EXAMPLE.COM

    Keytab File Name: /tmp/machine1.HTTP.keytab

    Kerberos Realm: ISQA.EXAMPLE.COM

    Kerberos Server Name:

    Return Principal with Domain Name: false

    Authentication Level: 22

    Note –

    If you are using Windows 2003 or Windows 2003 Service Packs,, use the following ktpass command syntax:

    ktpass /out filename /mapuser username /princ HTTP/hostname.domainname 
         /crypto encryptiontype /rndpass /ptype principaltype /target domainname

    For example:

    ktpass /out demo.HTTP.keytab /mapuser http 
    /princ HTTP/ /crypto RC4-HMAC-NT 
    /rndpass /ptype KRB5_NT_PRINCIPAL /target IDENTITY.SUN.COM

    For syntax definitions, see KTPASS Syntax.

  5. Restart the server.

ProcedureTo Reduce the Size of a Kerberos Ticket

All Active Directory groups to which a user belongs are encoded within an issued Kerberos ticket, increasing the size of the HTTP header. Choose one of the following options to reduce the ticket's size.

  1. Increase the default maximum header size of the web container being used.

    For example, when using Glassfish replace:

    <connection-pool max-pending-count="4096" 
    queue-size-in-bytes="4096" receive-buffer-size-in-bytes="4096" 


    <connection-pool max-pending-count="4096" 
    queue-size-in-bytes="65536" receive-buffer-size-in-bytes="65536" 
  2. Disable the PAC for the OpenSSO service account

    This is the Microsoft extension to Kerberos that contains the Active Directory groups. See

ProcedureTo Set Up Internet Explorer

  1. In the Tool menu, go to Internet Options > Advanced/Security > Security.

  2. Select the Integrated Windows Authentication option.

  3. Go to Security > Local Internet.

    1. Select Custom Level. In the User Authentication/Logon panel, select the Automatic Logon Only in Intranet Zone option.

    2. Go to Sites and select all of the options.

    3. Click Advanced and add the OpenSSO Enterprise to the local zone (if it is not added already).


If you are using Microsoft Internet Explorer 6.x for Windows Desktop SSO authentication and the browser does not have access to a Kerberos/SPNEGO token for the user that matches the (KDC) realm defined in the Windows Desktop SSO module, the browser will behave incorrectly for other modules after Windows Desktop SSO authentication fails. The direct cause of the problem is that after Windows Desktop SSO fails, the browser becomes incapable of passing all authentication module callbacks to OpenSSO Enterprise until the browser is restarted. Therefore all the modules coming after Windows Desktop SSO will fail due to null user credentials. See;en-us;308074 for related information.

Note –

As of this release, this restriction has been fixed by Microsoft. For more information, see

Installing a Samba Client for Windows NT Authentication

To activate the Windows NT authentication module, Samba Client 3.x must be downloaded and installed. Samba is a file and print server for blending Windows and UNIX machines without requiring a separate Windows NT/2003 Server. See To Install the Samba Client for more information.

Tip –

Red Hat Linux ships with a Samba client located in the /usr/bin directory.

ProcedureTo Install the Samba Client

  1. Download the Samba software from

  2. Install the software to the OpenSSO-Deploy-base/OpenSSO-deploy-uri/bin directory.

  3. (Optional) When running OpenSSO Enterprise on AIX, follow this sub procedure.

    See the Samba README for the reasons this startup script might be necessary for your deployment.

    1. Create a smbclient wrapper script.

      For example:

      unset LIBPATH
      /usr/bin/smbclient $@
    2. Copy the wrapper script to the OpenSSO-Deploy-base/OpenSSO-deploy-uri//bin directory.

  4. (Optional) Set multiple interfaces by configuration of the smb.conf file which is passed to the smbclient.

Configuring Authentication Modules

Before defining an authentication process, instances of the authentication modules to be used in the process must be added in the appropriate realm. Before adding an authentication module instance to a realm, default global values can be defined which will then be carried over to any instance created from the module. Once the module instance is added to the realm, you can modify the values specifically for that realm. See Authentication Modules for information about each authentication module.

ProcedureTo Define Global Values for an Authentication Module

  1. Log in to the OpenSSO Enterprise console as the administrator.

    By default, amadmin.

  2. Click the Configuration tab.

  3. Under Authentication, select the module you want to modify.

  4. Modify the values for the appropriate properties and click Save.

    The default values defined globally will be assigned per realm or sub realm when the authentication module is added. See To Add an Authentication Module Instance to a Realm or Sub Realm.

ProcedureTo Add an Authentication Module Instance to a Realm or Sub Realm

Authentication module instances can be expressly accessed for authentication (outside of an authentication chain) by using the module authentication type. See Module Authentication for more information.

  1. Log in to the OpenSSO Enterprise console as the administrator.

    By default, amadmin.

  2. Click the Access Control tab.

  3. Click the name of the realm under which you are adding an authentication module instance.

  4. Select the Authentication tab.

  5. Click New under Module Instances.

  6. Enter a Name for the module instance.

    The name must be unique.

  7. Select the Type of authentication module.

  8. Click OK.

  9. Click the name of the new instance to edit the properties for this realm.

    See Authentication in Sun OpenSSO Enterprise 8.0 Administration Reference, or click Help for definitions of the properties.

  10. Click Save.

    Repeat these steps to add multiple authentication module instances.

Creating Authentication Chains

An authentication chain is a configured authentication process in which a user must pass credentials to all module instances defined in it. Authentication chains are used to define the authentication process for all authentication types except Module. See Initiating the Authentication Type for more information.

Note –

Authentication chaining is achieved using the Java Authentication and Authorization Service (JAAS) framework integrated with the Authentication Service.

ProcedureTo Create an Authentication Chain

Before You Begin

Instances of all authentication modules to be used in the authentication chain must first be added to the realm under which you are creating the chain. See Configuring Authentication Modules for instructions.

  1. Log in to the OpenSSO Enterprise console as the administrator.

    By default, amadmin.

  2. Click the Access Control tab.

  3. Click the name of the realm under which you are creating a new authentication chain.

  4. Select the Authentication tab.

  5. Click New under Authentication Chaining.

  6. Enter a name for the authentication chain and click OK.

    The chain's properties page is displayed.

  7. Click Add to include the desired authentication module instance(s).

    A drop down list of authentication modules instantiated in the realm is displayed.

  8. Select the desired authentication module instance from the Instance list.

  9. Select the appropriate criteria for the module instance from the Criteria list.

    These flags establish the enforcement criteria for the module instance within a chain. There is a hierarchy for enforcement: REQUIRED is the highest and OPTIONAL is the lowest. More information can be found in the class document.


    Successful authentication to this module instance is required for the authentication process to succeed. If authentication to any REQUIRED module instances defined in a chain fails, authentication will fail. The authentication process will continue through the authentication chain whether authentication to the REQUIRED module instance succeeds or fails.


    Successful authentication to this module instance is required to proceed through the authentication chain. If authentication is successful, the authentication process moves to the next module instance in the authentication chain. If authentication fails, the chain is broken, control returns to the Authentication Service, and the user is not authenticated.


    Successful authentication to this module is not required but, if authentication does succeed, the user is authenticated and the authentication process will not continue through the authentication chain. If authentication to a SUFFICIENT module instance fails, the authentication process continues through the module instances in the authentication chain.


    Successful authentication to this module instance is not required but, whether it succeeds or fails, the authentication process continues through the module instances in the authentication chain.

  10. Enter options for the chain.

    This enables additional options to be defined for the module as a key=value pair. For example, if the authentication module supports debugging, enter debug=true. Multiple options are separated by a space. More information can be found in the class document.

  11. (Optional) Add values for the following attributes.

    Successful Login URL

    Specify a URL that the user will be redirected to upon successful authentication.

    Failed Login URL

    Specify a URL that the user will be redirected to upon failed authentication.

    Post Authentication Processing Class

    Specify the name of a Java class used to customize any post authentication processes (successful or failed).

  12. Click Save.