Chapter 3 Using Authentication Services (Tasks)

The first part of this chapter provides information about the Diffie-Hellman authentication mechanism that can be used with Secure RPC. The second part covers the Pluggable Authentication Module (PAM) framework. PAM provides a method to “plug-in” authentication services and provides support for multiple authentication services.

This is a list of the step-by-step instructions in this chapter.

• Overview of Secure RPC

• Administering Diffie-Hellman Authentication

• PAM (Overview)

• PAM (Tasks)

• PAM (Reference)

Overview of Secure RPC

Secure RPC is an authentication method that authenticates both the host and the user who is making a request for a service. Secure RPC uses the Diffie-Hellman authentication mechanism. This authentication mechanism uses DES encryption. Applications that use Secure RPC include NFS and the NIS+ name service.

NFS Services and Secure RPC

NFS enables several hosts to share files over the network. Under the NFS service, a server holds the data and resources for several clients. The clients have access to the file systems that the server shares with the clients. Users who are logged in to the client machines can access the file systems by mounting them from the server. To the user on the client machine, it appears as if the files are local to the client. One of the most common uses of NFS is to allow systems to be installed in offices, while keeping all user files in a central location. Some features of the NFS service, such as the mount -nosuid option, can be used to prohibit the opening of devices, as well as file systems, by unauthorized users.

The NFS service uses Secure RPC to authenticate users who make requests over the network. This process is known as Secure NFS. The authentication mechanism, AUTH_DH, uses DES encryption with Diffie-Hellman authentication to ensure authorized access. The AUTH_DH mechanism has also been called AUTH_DES.

• How to set up and administer Secure NFS in “Administering the Secure NFS System” in System Administration Guide: Resource Management and Network Services

• How to set up the NIS+ tables and enter names in the cred table in System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP)

• An outline of the steps that are involved in RPC authentication in Implementation of Diffie-Hellman Authentication

DES Encryption

The Data Encryption Standard (DES) encryption functions use a 56-bit key to encrypt data. If two credential users (or principals) know the same DES key, they can communicate in private, by using the key to encipher and decipher text. DES is a relatively fast encryption mechanism. A DES chip makes the encryption even faster. However, if the chip is not present, a software implementation is substituted.

The risk of using just the DES key is that an intruder can collect enough cipher-text messages that were encrypted with the same key to be able to discover the key and decipher the messages. For this reason, security systems such as Secure NFS change the keys frequently.

Kerberos Authentication

Kerberos is an authentication system that was developed at MIT. Encryption in Kerberos is based on DES. Kerberos V4 support is no longer supplied as part of Secure RPC. However, a client-side implementation of Kerberos V5, which uses RPCSEC_GSS, is included with this release. For more information see Chapter 6, Introduction to SEAM.

Diffie-Hellman Authentication

The Diffie-Hellman (DH) method of authenticating a user is nontrivial for an intruder to crack. The client and the server each have their own private key (sometimes called a secret key) which they use together with the public key to devise a common key. They use the common key to communicate with each other, by using an agreed-upon encryption/decryption function (such as DES). This method was identified as DES authentication in previous Solaris releases.

Authentication is based on the ability of the sending system to use the common key to encrypt the current time, which the receiving system can decrypt and check against its current time. Make sure that you synchronize the time on the client and the server.

The public keys and private keys are stored in an NIS or NIS+ database. NIS stores the keys in the publickey map. NIS+ stores the keys in the cred table. These files contain the public key and the private key for all potential users.

The system administrator is responsible for setting up NIS maps or NIS+ tables, and generating a public key and a private key for each user. The private key is stored in encrypted form with the user's password. This process makes the private key known only to the user.

Implementation of Diffie-Hellman Authentication

This section describes the series of transactions in a client-server session that use DH authorization (AUTH_DH).

Generating the Public Keys and Secret Keys

Sometime prior to a transaction, the administrator runs either the newkey or nisaddcred command to generate a public key and a secret key. Each user has a unique public key and secret key. The public key is stored in a public database. The secret key is stored in encrypted form in the same database. To change the key pair, use the chkey command.

Running the keylogin Command

Normally, the login password is identical to the secure RPC password. In this case, the keylogin command is not required. However, if the passwords are different, the users have to log in, and then run a keylogin command explicitly.

The keylogin command prompts the user for a secure RPC password and uses the password to decrypt the secret key. The keylogin command then passes the decrypted secret key to a program called the keyserver. The keyserver is an RPC service with a local instance on every computer. The keyserver saves the decrypted secret key and waits for the user to initiate a secure RPC transaction with a server.

If both the login password and the RPC password are the same, the login process passes the secret key to the keyserver. If the passwords are required to be different and the user must always run the keylogin command, then the keylogin command can be included in the user's environment configuration file, such as the ~/.login, ~/.cshrc, or ~/.profile file. Then, the keylogin command runs automatically whenever the user logs in.

Generating the Conversation Key

When the user initiates a transaction with a server, the following occurs:

1. The keyserver randomly generates a conversation key.

2. The kernel uses the conversation key to encrypt the client's time stamp (among other things).

3. The keyserver looks up the server's public key in the public key database (see the publickey(4) man page).

4. The keyserver uses the client's secret key and the server's public key to create a common key.

5. The keyserver encrypts the conversation key with the common key.

First Contact With the Server

The transmission, which includes the encrypted time stamp and the encrypted conversation key, is then sent to the server. The transmission includes a credential and a verifier. The credential contains three components:

• The client's net name

• The conversation key, which is encrypted with the common key

• A “window,” which is encrypted with the conversation key

The window is the difference in time that the client says should be allowed between the server's clock and the client's time stamp. If the difference between the server's clock and the time stamp is greater than the window, the server rejects the client's request. Under normal circumstances, this rejection will not happen, because the client first synchronizes with the server before starting the RPC session.

The client's verifier contains the following:

• The encrypted time stamp

• An encrypted verifier of the specified window, which is decremented by 1

The window verifier is needed in case somebody wants to impersonate a user and writes a program that, instead of filling in the encrypted fields of the credential and verifier, just stuffs in random bits. The server will decrypt the conversation key into some random key, and use it to try to decrypt the window and the time stamp. The result will be random numbers. After a few thousand trials, however, there is a good chance that the random window/time stamp pair will pass the authentication system. The window verifier makes guessing the right credential much more difficult.

Decrypting the Conversation Key

When the server receives the transmission from the client, the following occurs:

1. The keyserver that is local to the server looks up the client's public key in the public key database.

2. The keyserver uses the client's public key and the server's secret key to deduce the common key, which is the same common key that is computed by the client. Only the server and the client can calculate the common key because the calculation requires knowing one of the secret keys.

3. The kernel uses the common key to decrypt the conversation key.

4. The kernel calls the keyserver to decrypt the client's time stamp with the decrypted conversation key.

Storing Information on the Server

After the server decrypts the client's time stamp, it stores four items of information in a credential table:

• The client's computer name

• The conversation key

• The window

• The client's time stamp

The server stores the first three items for future use. The server stores the time stamp to protect against replays. The server accepts only time stamps that are chronologically greater than the last time stamp seen, so any replayed transactions are guaranteed to be rejected.

Implicit in these procedures is the name of the caller, who must be authenticated in some manner. The keyserver cannot use DES authentication to authenticate the caller because it would create a deadlock. To solve this problem, the keyserver stores the secret keys by user ID (UID) and grants requests only to local root processes.

Returning the Verifier to the Client

The server returns a verifier to the client, which includes the following:

• The index ID, which the server records in its credential cache

• The client's time stamp minus 1, which is encrypted by the conversation key

The reason for subtracting 1 from the time stamp is to ensure that the time stamp is invalid and that it cannot be reused as a client verifier.

Client Authenticates the Server

The client receives the verifier and authenticates the server. The client knows that only the server could have sent the verifier because only the server knows what time stamp the client sent.

Additional Transactions

With every transaction after the first transaction, the client returns the index ID to the server in its second transaction and sends another encrypted time stamp. The server sends back the client's time stamp minus 1, which is encrypted by the conversation key.

Administering Diffie-Hellman Authentication

A system administrator can implement policies that help secure the network. The level of security that is required differs with each site. This section provides instructions for some tasks that are associated with network security.

How to Restart the Keyserver

1. Become superuser or assume an equivalent role.

2. Verify whether the keyserv daemon (the keyserver) is running.

   # ps -ef | grep keyserv root 100 1 16 Apr 11 ? 0:00 /usr/sbin/keyserv root 2215 2211 5 09:57:28 pts/0 0:00 grep keyserv

3. Start the keyserver if it isn't running.

   # /usr/sbin/keyserv

How to Set Up a Key in NIS+ Credentials for Diffie-Hellman Authentication

For detailed description of NIS+ security, see System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP).

1. Become superuser or assume an equivalent role.

2. Edit the /etc/nsswitch.conf file, and add the following line:

   publickey: nisplus

3. Initialize the NIS+ client.

   # nisinit -cH hostname

   hostname is the name of a trusted NIS+ server that contains an entry in its tables for the client machine.

4. Add the client to the cred table by typing the following commands:

   # nisaddcred local # nisaddcred des

5. Verify the setup by using the keylogin command.

   If you are prompted for a password, the procedure has succeeded.

Example—Setting Up a New Key for root on a NIS+ Client

The following example uses the host pluto to set up earth as an NIS+ client. You can ignore the warnings. The keylogin command is accepted, verifying that earth is correctly set up as a secure NIS+ client.

# nisinit -cH pluto
NIS Server/Client setup utility.
This machine is in the North.Abc.COM. directory.
Setting up NIS+ client ...
All done.
# nisaddcred local
# nisaddcred des 
DES principal name : unix.earth@North.Abc.COM
Adding new key for unix.earth@North.Abc.Com (earth.North.Abc.COM.)
 
Network password: xxx Press Return
Warning, password differs from login password.
Retype password: xxx Press Return
 
# keylogin
Password:
#

How to Set Up a New User Key That Uses NIS+ Credentials for Diffie-Hellman Authentication

1. Add the user to the cred table on the root master server by typing the following command:

   # nisaddcred -p unix.UID@domain-name -P username.domain-name. des

   Note that, in this case, the username.domain-name must end with a dot (.)

2. Verify the setup by logging in as the client and typing the keylogin command.

Example—Setting Up a New Key for an NIS+ User

The following example shows DES security authorization being given to a user that is named george.

# nisaddcred -p unix.1234@North.Abc.com -P george.North.Abc.COM. des
DES principal name : unix.1234@North.Abc.COM
Adding new key for unix.1234@North.Abc.COM (george.North.Abc.COM.)
 
Password:
Retype password:
 
# rlogin rootmaster -l george
# keylogin
Password:
#

How to Set Up a root Key by Using NIS Credentials With Diffie-Hellman Authentication

1. Become superuser on the client or assume an equivalent role.

2. Edit the /etc/nsswitch.conf file, and add the following line:

   publickey: nis

3. Create a new key pair by using the newkey command.

   # newkey -h hostname

   hostname is the name of the client.

Example—Setting Up an NIS+ Client to Use Diffie-Hellman Security

The following example shows how to set up earth as a secure NIS client.

# newkey -h earth
Adding new key for unix.earth@North.Abc.COM
New Password:
Retype password:
Please wait for the database to get updated...
Your new key has been successfully stored away.
#

How to Create a New User Key That Uses NIS Credentials with Diffie-Hellman Authentication

1. Log in to the server as superuser or assume an equivalent role.

   Only system administrators, when logged in to the NIS+ server, can generate a new key for a user.

2. Create a new key for a user.

   # newkey -u username

   username is the name of the user. The system prompts for a password. You can type a generic password. The private key is stored in an encrypted form using the generic password.

   # newkey -u george Adding new key for unix.12345@Abc.North.Acme.COM New Password: Retype password: Please wait for the database to get updated... Your new key has been successfully stored away. #

3. Tell the user to log in and type the chkey -p command.

   This command allows the user to re-encrypt his or her private key with a password known only to the user.

   earth% chkey -p Updating nis publickey database. Reencrypting key for unix.12345@Abc.North.Acme.COM Please enter the Secure-RPC password for george: Please enter the login password for george: Sending key change request to pluto... #

   ---

   Note –

   The chkey command can be used to create a new key-pair for a user.

   ---

How to Share and Mount Files With Diffie-Hellman Authentication

Prerequisite

The Diffie-Hellman publickey authentication must be enabled on the network. See How to Set Up a Key in NIS+ Credentials for Diffie-Hellman Authentication and How to Set Up a root Key by Using NIS Credentials With Diffie-Hellman Authentication.

To share a file system with Diffie-Hellman authentication:

1. Become superuser or assume an equivalent role.

2. Share the file system with Diffie-Hellman authentication.

   # share -F nfs -o sec=dh /filesystem

To mount a file system with Diffie-Hellman authentication:

1. Become superuser or assume an equivalent role.

2. Mount the file system with Diffie-Hellman authentication.

   # mount -F nfs -o sec=dh server:resource mountpoint

   The -o sec=dh option mounts the file system with AUTH_DH authentication.

PAM (Overview)

The Pluggable Authentication Module (PAM) framework lets you “plug in” new authentication technologies without changing system entry services such as login, ftp, telnet, and so on. You can also use PAM to integrate UNIX login with other security mechanisms like DCE or Kerberos. Mechanisms for account, session, and password management can also be “plugged in” by using this framework.

Benefits of Using PAM

The PAM framework allows you to choose any combination of system entry services (ftp, login, telnet, or rsh, for example) for user authentication. Some benefits that PAM provides are as follows:

• Flexible configuration policy

  • Per application authentication policy

  • The ability to choose a default authentication mechanism

  • Multiple passwords on high-security systems

• Ease of use for the end user

  • No retyping of passwords if they are the same for different mechanisms

  • The ability to use a single password for multiple authentication methods with the password-mapping feature, even if the passwords that are associated with each authentication method are different

  • The ability to prompt the user for passwords for multiple authentication methods without having the user enter multiple commands

• The ability to pass optional parameters to the user authentication services

PAM Components

The PAM software consists of a library, several modules, and a configuration file. New versions of several commands or daemons that take advantage of the PAM interfaces are also included.

The following figure below illustrates the relationship between the applications, the PAM library, the pam.conf file, and the PAM modules.

Figure 3–1 How PAM Works

The applications (ftp, telnet, and login) use the PAM library to access the appropriate module. The pam.conf file defines which modules to use, and in what order they are to be used with each application. Responses from the modules are passed back through the library to the application.

The following sections describe the relationship between the PAM components and the applications.

PAM Library

The PAM library, /usr/lib/libpam, provides the framework to load the appropriate modules and manage the stacking process. The PAM library provides a generic structure to which all of the modules can plug in.

Stacking Feature

The PAM framework provides a method for authenticating users with multiple services by using stacking. Depending on the configuration, the user can be prompted for passwords for each authentication method. The order in which the authentication services are used is determined through the PAM configuration file.

Password-Mapping Feature

The stacking feature can require that a user remember several passwords. With the password-mapping feature, the primary password is used to decrypt the other passwords so that the user doesn't need to remember or enter multiple passwords. The other option is to synchronize the passwords across each authentication mechanism. Note that this strategy could increase the security risk, because the security of each mechanism is limited by the least secure password method that is used in the stack.

Changes to PAM for the Solaris 9 Release

The Solaris 9 release includes several enhancements to the PAM service. The following list highlights the most important changes:

• To accommodate proper stacking, the pam_unix module is broken into single service modules. These modules provide the same capabilities as in the existing module. The capabilities are provided by the following modules: pam_authtok_get, pam_authtok_check, pam_authtok_store, pam_unix_auth, pam_dhkeys and pam_passwd_auth. See PAM Modules for information about the new modules.

• New PAM services are included: cron, dtsession, ppp and ssh. See Valid Service Names for information about the new services.

• The PAM configuration file was updated to include the new modules and services. See Generic pam.conf File for information about the configuration file.

PAM (Tasks)

This section discusses some tasks that might be required to make the PAM framework fully functional. In particular, you should be aware of some security issues that are associated with the PAM configuration file.

PAM (Task Map)

Planning for PAM

When you are deciding how best to use PAM in your environment, start by focusing on these issues:

• Determine what your needs are, especially which modules you should select.

• Identify the services that need special attention. Use OTHER if appropriate.

• Decide on the order in which the modules should be run.

• Select the control flag for each module.

• Choose any options that are necessary for each module.

Here are some suggestions to consider before you change the PAM configuration file:

• Use the OTHER entry for each module type so that every application does not have to be included.

• Make sure to consider the security implications of the sufficient and optional control flags.

• Review the man pages that are associated with the modules to understand how each module functions, what options are available, and the interactions between stacked modules.

  ---

  Caution –

  If the PAM configuration file is misconfigured or becomes corrupted, it is possible that even superuser would be unable to log in. Since the sulogin command does not use PAM, superuser would then be required to boot the machine into single-user mode and fix the problem.

  ---

  After you change the /etc/pam.conf file, review it as much as possible while you are still logged in as superuser. Test all the commands that might have been affected by your changes. For example, if you added a new module to the telnet service, use the telnet command and verify that the changes you made make the service behave as expected.

How to Add a PAM Module

1. Become superuser or assume an equivalent role.

2. Determine which control flags and other options should be used.

   Refer to PAM Modules information on the modules.

3. Copy the new module to /usr/lib/security/sparcv9.

   In the Solaris 8 release, the module should be copied to /usr/lib/security.

4. Set the permissions so that the module file is owned by root and that permissions are 555.

5. Edit the PAM configuration file, /etc/pam.conf, and add this module to the appropriate services.

Verification

It is very important to do some testing before the system is rebooted in case the configuration file is misconfigured. Run rlogin, su, and telnet before you reboot the system. If the service is a daemon that is spawned only once when the system is booted, it might be necessary to reboot the system before you can verify that the module has been added.

How to Prevent Unauthorized Access From Remote Systems With PAM

Remove the rlogin auth rhosts_auth.so.1 entry from the PAM configuration file. This step prevents the reading of the ~/.rhosts files during an rlogin session and therefore prevents unauthenticated access to the local system from remote systems. All rlogin access requires a password, regardless of the presence or contents of any ~/.rhosts or /etc/hosts.equiv files.

To prevent other unauthenticated access to the ~/.rhosts files, remember to disable the rsh service. The best way to disable a service is to remove the service entry from the /etc/inetd.conf file. Changing the PAM configuration file does not prevent the service from being started.

How to Initiate PAM Error Reporting

1. Edit the /etc/syslog.conf file to add any of the following entries for PAM error reporting:

   • auth.alert - Messages about conditions that should be fixed immediately

   • auth.crit - Critical messages

   • auth.err - Error messages

   • auth.info - Informational messages

   • auth.debug - Debugging messages

2. Restart the syslog daemon, or send a SIGHUP signal to it to activate the PAM error reporting.

Example—Initiating PAM Error Reporting

In the following example, all alert messages are displayed on the console. Critical messages are mailed to root. Informational and debug messages are added to the /var/log/pamlog file.

auth.alert	/dev/console
auth.crit	'root'
auth.info;auth.debug	/var/log/pamlog

Each line in the log contains a time stamp, the name of the system that generated the message, and the message itself. The pamlog file is capable of logging a large amount of information.

PAM (Reference)

PAM uses run-time pluggable modules to provide authentication for system entry services. These modules are broken into four different types, based on their function:

• authentication

• account management

• session management

• password management

A stacking feature is provided to let you authenticate users through multiple services. Also provided is a password-mapping feature to not require that users remember multiple passwords.

PAM Modules

Each PAM module implements a specific mechanism. When you set up PAM authentication, you need to specify both the module and the module type, which defines what the module does. More than one module type (auth, account, session, or password) can be associated with each module.

The following table describes each PAM module, and includes the module name and the module file name. The path of each module is determined by the instruction set that is available in the Solaris release that is installed. The default path to the modules is /usr/lib/security/$ISA. The value for $ISA could be sparc or i386. See the isalist(5) man page for more information.

For security reasons, these module files must be owned by root and must not be writable through group or other permissions. If the file is not owned by root, PAM will not load the module.

PAM Module Types

It is important to understand the PAM module types because they define the interface to the module. Here are the four types of run-time PAM modules:

• The authentication modules provide authentication for the users and allow for credentials to be set, refreshed, or destroyed. They provide a valuable administration tool for user identification.

• The account modules check for password aging, account expiration, and access hour restrictions. After the user is identified through the authentication modules, the account modules determine if the user should be given access.

• The session modules manage the opening and the closing of an authentication session. They can log activity or provide for clean-up after the session is over.

• The password modules allow for changes to the actual password.

PAM Configuration File

The PAM configuration file, /etc/pam.conf, determines the authentication services to be used, and in what order they are used. This file can be edited to select authentication mechanisms for each system entry application.

PAM Configuration File Syntax

The PAM configuration file consists of entries with the following syntax:

service_name module_type control_flag module_path module_options

You can add comments to the pam.conf file by starting the line with a # (pound sign). Use white spaces or tabs to delimit the fields.

An entry in the PAM configuration file is ignored if one of the following conditions exist: the line has less than four fields, an invalid value is given for module_type or control_flag, or the named module does not exist.

Valid Service Names

The following table lists some valid service names, the module types that can be used with that service, and the daemon or command that is associated with the service name.

Not all module types are appropriate for each service. For example, the password module type is appropriate for only the passwd command. Also, since the passwd command is not concerned with authentication, there is no auth module type associated with it.

Control Flags

To determine the continuation or failure behavior from a module during the authentication process, you must select one of four control flags for each entry in the PAM configuration file, /etc/pam.conf. The control flags indicate how a successful attempt or a failed attempt through each module is handled. Even though these flags apply to all module types, the following explanation assumes that these flags are being used for authentication modules. The control flags are as follows:

• required - With this control flag, the module must return success in order to have the overall result be successful.

  If all modules are flagged as required, then authentication through all modules must succeed for the user to be authenticated.

  If some modules fail, then an error value from the first failed module is reported.

  If a failure occurs for a module that is flagged as required, all modules in the stack are still tried, but failure is returned.

  If none of the modules are flagged as required, then at least one entry for that service must succeed for the user to be authenticated.

• requisite - With this control flag, the module must return success for additional authentication to occur.

  If a failure occurs for a module that is flagged as requisite, an error is immediately returned to the application, and no additional authentication is done. If the stack does not include prior modules flagged as required that failed, then the error from this module is returned. If an earlier module flagged as required has failed, the error message from the required module is returned.

• optional - If a module with this control flag fails, the overall result can be successful if another module in this stack returns success.

  The optional control flag should be used when successful authentication in the stack is enough for a user to be authenticated. This flag should only be used if it is not important for this particular mechanism to succeed.

  If your users need to have permission associated with a specific mechanism to get their work done, then you should not label it as optional.

• sufficient - If a module with this control flag is successful, skip the remaining modules in the stack, even if they are flagged as required.

  The sufficient control flag indicates that one successful authentication will be enough for the user to be granted access.

More information about these control flags is provided in the following section, which describes the default /etc/pam.conf file.

Generic pam.conf File

The generic /etc/pam.conf file specifies the following actions:

1. When the login command is run, authentication must succeed for the pam_authtok_get, pam_dhkeys, pam_auth_unix and the pam_dial_auth modules.

2. For the rlogin command, authentication through the pam_authtok_get, pam_dhkeys, and pam_auth_unix modules must succeed, if authentication through pam_rhost_auth fails.

3. The sufficient control flag indicates that for the rlogin command the successful authentication that is provided by the pam_rhost_auth module is sufficient, and the next entry will be ignored.

4. Most of the other commands that require authentication require successful authentication through the pam_authtok_get, pam_dhkeys, and pam_auth_unix modules.

5. For the rsh command, authentication through the pam_rhost_auth module is flagged as sufficient. No other authentication is required if authentication through the pam_rhost_auth module succeeds.

The OTHER service name allows a default to be set for any other commands that require authentication that are not included in the file. The OTHER option makes it easier to administer the file, since many commands that are using the same module can be covered by using only one entry. Also, the OTHER service name, when used as a “catch-all,” can ensure that each access is covered by one module. By convention, the OTHER entry is included at the bottom of the section for each module type.

Normally, the entry for the module_path is “root-relative.” If the file name you enter for module_path does not begin with a slash (/), the path /usr/lib/security/$ISA is prepended to the file name. A full path name must be used for modules that are located in other directories. The values for the module_options can be found in the man pages for the module. For example, the UNIX module is covered in the pam_unix(5) man page.

login   auth required           pam_authtok_get.so.1
login   auth required           pam_dhkeys.so.1
login   auth required           pam_unix_auth.so.1
login   auth required           pam_dial_auth.so.1

In this example, the login service specifies authentication through all four authentication modules. A login command fails if any of the modules return an error.