System Administration Guide: Security Services

Chapter 14 Managing System Security (Overview)

Keeping a system's information secure is an important system administration responsibility. This chapter provides overview information about managing system security at the file, system, and network level.

This is a list of the overview information in this chapter.

Controlling Access to a Computer System

At the file level, the SunOS operating system provides some standard security features that you can use to protect files, directories, and devices. At the system and network levels, the security issues are mostly the same. In the workplace, a number of systems that are connected to a server can be thought of as one large multifaceted system. The system administrator is responsible for the security of this larger system or network. Not only is it important to defend the network from outsiders who are trying to gain access to the network, but it is also important to ensure the integrity of the data on the systems within the network.

The first line of security defense is to control access to your system. You can control and monitor system access by doing the following:

Maintaining Physical Site Security

To control access to your system, you must maintain the physical security of your computing environment. For instance, if a system is logged in and left unattended, anyone who can use that system can gain access to the operating system and the network. You need to be aware of your computer's surroundings and physically protect it from unauthorized access.

Maintaining Login Control

You also must restrict unauthorized logins to a system or the network, which you can do through password and login control. All accounts on a system should have a password. An account without a password makes your entire network accessible to anyone who can guess a user name.

Solaris software restricts control of certain system devices to the user login account. Only a process that is running as superuser or console user can access a system mouse, keyboard, frame buffer, or audio device unless the /etc/logindevperm file is edited. For more information, see logindevperm(4).

Restricting Access to Data in Files

After you have established login restrictions, you can control access to the data on your system. You might want to allow some users to read some files, and give other users permission to change or delete some files. You might have some data that you do not want anyone else to see. Chapter 15, Securing Files (Tasks) discusses how to set file permissions.

Maintaining Network Control

Computers are often part of a configuration of systems called a network. A network allows connected systems to exchange information and access data and other resources that are available from systems connected to the network. Networking has created a powerful and sophisticated way of computing. However, networking has also jeopardized computer security.

For instance, within a network of computers, individual systems are open to allow sharing of information. Also, because many people have access to the network, there is more chance for allowing unwanted access, especially through user error (for example, through a poor use of passwords).

Monitoring System Usage

As system administrator, you need to monitor system activity, being aware of all aspects of your systems, including the following:

With this kind of knowledge, you can use the available tools to audit system use and monitor the activities of individual users. Monitoring is very useful when there is a suspected breach in security.

Setting the Correct Path

It is important to set your path variable correctly. Otherwise, you can accidentally run a program that was introduced by someone else that harms your data or your system. This kind of program, which creates a security hazard, is referred to as a “Trojan horse.” For example, a substitute su program could be placed in a public directory where you, as system administrator, might run it. Such a script would look just like the regular su command. Since the script removes itself after execution, it is hard to tell that you have actually run a Trojan horse.

The path variable is automatically set at login time through the startup files: .login, .profile, and .cshrc. Setting up the user search path so that the current directory (.) comes last prevents you or your users from running this type of Trojan horse. The path variable for superuser should not include the current directory at all. The Automated Security Enhancement Tool (ASET) examines the startup files to ensure that the path variable is set up correctly and that it does not contain a dot (.) entry.

Securing Files

Since the SunOS operating system is a multiuser system, file system security is the most basic, and most important, security risk on a system. You can use both the traditional UNIX file protection or the more secure access control lists (ACLs) to protect your files.

Also, many executable programs have to be run as root (that is, as superuser) to work properly. These executables run with the user ID set to 0 (setuid=0). Anyone who is running these programs runs them with the root ID, which creates a potential security problem if the programs are not written with security in mind.

Except for the executables that are shipped with the setuid bit set to root, you should disallow the use of setuid programs, or at least restrict them and keep them to a minimum.

Installing a Firewall

Another way to protect your network is to use a firewall or secure gateway system. A firewall is a dedicated system that separates two networks, each of which approaches the other as untrusted. You should consider this setup as mandatory between your internal network and any external networks, such as the Internet, with which you want internal network users to communicate.

A firewall can also be useful between some internal networks. For example, the firewall or secure gateway computer will not send a packet between two networks unless the gateway computer is the origin or the destination address of the packet. A firewall should also be set up to forward packets for particular protocols only. For example, you can allow packets for transferring mail, but not those packets for the telnet or rlogin command. ASET, when run at high security, disables the forwarding of Internet Protocol (IP) packets.

Reporting Security Problems

If you experience a suspected security breach, you can contact the Computer Emergency Response Team/Coordination Center (CERT/CC). CERT/CC is a Defense Advanced Research Projects Agency (DARPA) funded project that is located at the Software Engineering Institute at Carnegie Mellon University. This agency can assist you with any security problems you are having. This agency can also direct you to other Computer Emergency Response Teams that might be more appropriate for your particular needs. You can call CERT/CC at its 24-hour hotline: (412) 268-7090, or contact the team by email at

File Security

The SunOS operating system is a multiuser system, which means that all the users who are logged in to a system can read and use files that belong to one another, as long as they have the file permissions to do so. Table 14–1 describes the commands for file system security. For step-by-step instructions on securing files, see Chapter 15, Securing Files (Tasks).

Commands for File System Security

This table describes the commands for monitoring and securing files and directories.

Table 14–1 Commands for File System Security



Man Page 


Lists the files in a directory and information about them. 



Changes the ownership of a file. 



Changes the group ownership of a file. 



Changes permissions on a file. You can use either symbolic mode (letters and symbols) or absolute mode (octal numbers) to change permissions on a file. 


File Encryption

By placing a sensitive file into an inaccessible directory (700 mode) and making the file unreadable by other users (600 mode), you will keep it secure in most cases. However, someone who guesses your password or the root password can read and write to that file. Also, the sensitive file is preserved on a backup tape every time you back up the system files to tape.

Fortunately, an additional layer of security is available to all SunOS system software users in the United States: the optional file encryption kit. The encryption kit includes the crypt command, which scrambles the data to disguise the text. For more information, see crypt(1).

Access Control Lists (ACLs)

ACLs (pronounced “ackkls”) can provide greater control over file permissions when the traditional UNIX file protection in the SunOS operating system is not sufficient. The traditional UNIX file protection provides read, write, and execute permissions for the three user classes: owner, group, and other. An ACL provides better file security by enabling you to define file permissions for the owner, owner's group, others, specific users and groups, and to define default permissions for each of those categories. For step–by–step instructions on using ACLs, see Using Access Control Lists (ACLs).

The following table lists the commands for administering ACLs on files or directories.

Table 14–2 ACL Commands



Man Page 


Sets, adds, modifies, and deletes ACL entries 



Displays ACL entries  


System Security

This section describes how to safeguard your system against unauthorized access, such as the following:

You can set up two security barriers on a system. The first security barrier is the login command. To cross this barrier and gain access to a system, a user must supply a user name and a corresponding password that is known by the local system or by the name service (LDAP, NIS, or NIS+).

The second security barrier is ensuring that the system files and programs can be changed or removed by superuser only. A would-be superuser must supply the root user name and its correct password.

Login Access Restrictions

When a user logs in to a system, the login command consults the appropriate database according to the information that is listed in the /etc/nsswitch.conf file. The entries in this file can include files (which designates the /etc files), nis (which designates the NIS database), ldap (which designates the LDAP directory service), and nisplus (which designates the NIS+ database). For a description of this file, see the nsswitch.conf(4) man page. For information about naming or directory services, see System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP) or System Administration Guide: Naming and Directory Services (FNS and NIS+).

The login command verifies the user name and password that were entered. If the user name is not in the password file or the password is not correct for the user name, the login command denies access to the system. When the user supplies a user name from the password file and the correct password for the user name, the system grants the user access to the system.

Special Logins

There are two common ways to access a system: by using a conventional user login, or by using the root login. In addition, a number of special system logins allow a user to perform administrative commands without using the root account. The administrator assigns passwords to these login accounts.

The following table lists some system login accounts and their uses. The system logins perform special functions, and each login has its own group identifier number (GID). Each of these logins should have its own password, which should be distributed on a need-to-know basis.

Table 14–3 System Logins

Login Account 





Has almost no restrictions and overrides all other logins, protections, and permissions. The root account has access to the entire system. The password for the root login should be very carefully protected. The root account owns most of the Solaris commands.



Controls background processing.  



Owns some of the Solaris commands. 



Owns many system files.  



Owns certain administrative files.  



Owns the object data files and spooled data files for the printer. 



Owns the object data files and spooled data files for UUCP, the UNIX-to-UNIX copy program. 



Is used by remote systems to log in to the system and start file transfers.  

You should also set the security of the eeprom command to require a password. See eeprom(1M) for more information.

Managing Password Information

When users log in to a system, they must enter both a user name and a password. Although logins are publicly known, passwords must be kept secret, known only to each user. You should ask your users to choose their passwords carefully, and they should change them often.

Passwords are initially created when you set up a user account. To maintain security on user accounts, you can set up password aging to force users to routinely change their passwords. You can also disable a user account by locking the password. For detailed information about setting up and maintaining passwords, see “Managing User Accounts and Groups (Overview)” in System Administration Guide: Basic Administration and the passwd(1) man page.

NIS+ Password File

If your network uses NIS+, the password information is kept in the NIS+ database. Information in the NIS+ database can be protected by restricting access to authorized users. You can use the passwd command to change a user's NIS+ password.

NIS Password File

If your network uses NIS, the password information is kept in the NIS password map. NIS does not support password aging. You can use the passwd command to change a user's NIS password.

/etc Files

If your network uses /etc files, the password information is kept in the system's /etc/passwd and /etc/shadow files. The user name and other information are kept in the password file /etc/passwd, while the encrypted password itself is kept in a separate shadow file, /etc/shadow. This security measure prevents a user from gaining access to the encrypted passwords. While the /etc/passwd file is available to anyone who can log in to a machine, only superuser can read the /etc/shadow file. You can use the passwd command to change a user's password on a local system.

LDAP Password Information

If your network uses LDAP, the password and shadow information is stored in the ou=people container of the LDAP directory tree. You can use the password -r ldap command to change a user's LDAP password.

Using the Restricted Shell

The standard shell allows a user to open files, execute commands, and so on. The restricted shell (rsh) can be used to limit the ability of a user to change directories and execute commands. The restricted shell is located in the /usr/lib directory. Note that the restricted shell is not the remote shell, which is /usr/sbin/rsh. The restricted shell differs from the standard shell in these ways:

The restricted shell allows the system administrator to limit a user's ability to stray into the system files, and is intended mainly to set up a user who needs to perform specific tasks. The restricted shell is not completely secure, however, and is only intended to keep unskilled users from getting into (or causing) trouble.

For information about the restricted shell, see the rsh(1M) man page.

A more secure alternative to the restricted shell is the Secure Shell (ssh), which enables users to securely access a remote host over an unsecured network. For information about using the Secure Shell, see Chapter 5, Secure Shell Administration (Reference).

Tracking Superuser (Root) Login

Your system requires a root password for superuser mode. In the default configuration, a user cannot remotely log in to a system as root. When logging in remotely, a user must log in as himself or herself and then use the su command to become root. This setup enables you to track who is using superuser privileges on your system.

Monitoring Who is Becoming Superuser or Other Users

You have to use the su command to change to another user, for example, if you want to become superuser. For security reasons, you can monitor who has been using the su command, especially those users who are trying to gain superuser access.

For detailed instructions, see How to Monitor Who Is Using the su Command.

Network Security

The more available access is across a network, the more advantageous it is for networked systems. However, free access and the sharing of data and resources create security problems. Network security is usually based on limiting or blocking operations from remote systems. The following figure describes the security restrictions that you can impose on remote operations.

Figure 14–1 Security Restrictions for Remote Operations

Diagram shows three ways to restrict access to remote systems: a firewall system, an authentication mechanism, and an authorization mechanism.

Firewall Systems

You can set up a firewall system to protect the resources in your network from outside access. A firewall system is a secure host that acts as a barrier between your internal network and outside networks.

The firewall has two functions. The firewall acts as a gateway that passes data between the networks, and it acts as a barrier that blocks the free passage of data to and from the network. The firewall requires a user on the internal network to log in to the firewall system to access hosts on remote networks. Similarly, a user on an outside network must log in to the firewall system before being granted access to a host on the internal network.

In addition, all electronic mail that is sent from the internal network is sent to the firewall system for transfer to a host on an external network. The firewall system receives all incoming electronic mail, and distributes it to the hosts on the internal network.

Caution – Caution –

A firewall prevents unauthorized users from accessing the hosts on your network. You should maintain strict and rigidly enforced security on the firewall, but security on other hosts on the network can be more relaxed. However, an intruder who can break into your firewall system can then gain access to all the other hosts on the internal network.

A firewall system should not have any trusted hosts. A trusted host is a host from which a user can log in without being required to type in a password. A firewall system should not share any of its file systems, or mount any file systems from other servers.

ASET can be used to make a system into a firewall, and to enforce high security on a firewall system, as described in Chapter 20, Using the Automated Security Enhancement Tool (Tasks).

Packet Smashing

Most local area networks transmit data between computers in blocks called packets. Through a procedure called packet smashing, unauthorized users can harm or destroy data. Packet smashing involves capturing the packets before they reach their destination, injecting arbitrary data into the contents, and then sending the packets back on their original course. On a local area network, packet smashing is impossible because packets reach all systems, including the server, at the same time. Packet smashing is possible on a gateway, however, so make sure that all gateways on the network are protected.

The most dangerous attacks are those attacks that affect the integrity of the data. Such attacks involve changing the contents of the packets or impersonating a user. Attacks that involve eavesdropping—recording conversations and replaying them later without impersonating a user—do not compromise data integrity. These attacks do affect privacy, however. You can protect the privacy of sensitive information by encrypting data that goes over the network.

Authentication and Authorization

Authentication is a way to restrict access to specific users when they access a remote system, which can be set up at both the system level or network level. Once a user gains access to a remote system, authorization is a way to restrict operations that the user can perform on the remote system. The following table lists the types of authentications and authorizations that can help protect your systems on the network against unauthorized use.

Table 14–4 Types of Authentication and Authorization for Remote Access



Where to Find Information 

LDAP and NIS+ 

The LDAP directory service and the NIS+ name service can provide both authentication and authorization at the network level. 

System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP) and System Administration Guide: Naming and Directory Services (FNS and NIS+)

Remote login commands 

The remote login commands (rlogin, rcp, ftp) enable users to log in to a remote system over the network and use its resources. If you are a “trusted host,” authentication is automatic. Otherwise, you are asked to authenticate yourself.

“Accessing Remote Systems (Tasks)” in System Administration Guide: Resource Management and Network Services

Secure RPC 

Secure RPC improves the security of network environments by authenticating users who make requests on remote systems. You can use either the UNIX, DES, or Kerberos authentication system for Secure RPC. 

Overview of Secure RPC


Secure RPC can also be used to provide additional security to the NFS environment, called Secure NFS. 

NFS Services and Secure RPC

DES encryption 

The Data Encryption Standard (DES) encryption functions use a 56-bit key to encrypt a secret key. 

DES Encryption

Diffie-Hellman authentication 

This authentication method is based on the ability of the sending system to use the common key to encrypt the current time, which the receiving system can decrypt and check against its current time. 

Diffie-Hellman Authentication


Kerberos uses DES encryption to authenticate a user when logging in to the system.  

Chapter 3, Using Authentication Services (Tasks)

Sharing Files

A network file server can control which files are available for sharing. A network file server can also control which clients have access to the files, and what type of access is permitted for those clients. In general, the file server can grant read and write access or read-only access either to all clients or to specific clients. Access control is specified when resources are made available with the share command.

A server can use the /etc/dfs/dfstab file to list the file systems that it makes available to clients on the network. For more information about sharing file systems, see “Automatic File-System Sharing” in System Administration Guide: Resource Management and Network Services.

Restricting Superuser (Root) Access

In general, superuser is not allowed root access to file systems that are shared across the network. Unless the server specifically grants superuser privileges, a user who is logged in as superuser on a client cannot gain root access to files that are remotely mounted on the client. The NFS system implements this strategy by changing the user of the requester to the user, nobody (user ID 60001). The access rights of user nobody are the same as those access rights that are given to the public or a user without credentials. For example, if the public has only execute permission for a file, then user nobody can only execute that file.

An NFS server can grant superuser privileges on a shared file system on a per-host basis by using the root=hostname option to the share command.

Using Privileged Ports

If you do not want to run Secure RPC, a possible substitute is the Solaris “privileged port” mechanism. A privileged port is assigned with a port number of less than 1024. After a client system has authenticated the client's credential, it builds a connection to the server by using the privileged port. The server then verifies the client credential by examining the connection's port number.

Non-Solaris clients, however, might be unable to communicate by using the privileged port. If they cannot, you see an error message similar to the following:

“Weak Authentication
NFS request from unprivileged port”

Using the Automated Security Enhancement Tool (ASET)

The ASET security package provides automated administration tools that enable you to control and monitor your system's security. You specify a security level (low, medium, or high) at which ASET will run. At each higher level, ASET's file-control functions increase to reduce file access and tighten your system security.

For more information, see Chapter 20, Using the Automated Security Enhancement Tool (Tasks).