test

NFS Administration Guide

Chapter 3 NFS Reference

This chapter provides an introduction to the NFS commands. This chapter also provides information about all of the pieces of the NFS environment and how these pieces work together.

NFS Files

You need several ASCII files to support NFS activities on any computer. Table 3-1 lists these files and their functions.

Table 3-1 NFS ASCII Files

File Name 

Function 

/etc/mnttab

Lists file systems that are currently mounted including automounted directories (see the mnttab(4) man page); do not edit this file.

/etc/netconfig

Lists the transport protocols; do not edit this file. 

/etc/nfssec.conf

Lists NFS security services; do not edit this file. 

/etc/rmtab

Lists file systems remotely mounted by NFS clients (see the rmtab(4) man page); do not edit this file.

/etc/vfstab

Defines file systems to be mounted locally (see the vfstab(4) man page).

/etc/default/fs

Lists the default file system type for local file systems. 

/etc/dfs/dfstab

Lists the local resources to be shared. 

/etc/dfs/fstypes

Lists the default file-system types for remote file systems. 

/etc/dfs/sharetab

Lists the resources (local and remote) that are shared (see the sharetab(4) man page); do not edit this file.

The first entry in /etc/dfs/fstypes is often used as the default file-system type for remote file systems. This entry defines the NFS file-system type as the default.

There is only one entry in /etc/default/fs: the default file-system type for local disks. You can determine the file system types that are supported on a client or server by checking the files in /kernel/fs.

NFS Daemons

To support NFS activities, several daemons are started when a system goes into run-level 3 or multiuser mode. Two of these daemons (mountd and nfsd) are run on systems that are NFS servers. The automatic startup of the server daemons depends on the existence of entries labeled with the NFS file-system type in /etc/dfs/sharetab.

The other two daemons (lockd and statd) are run on NFS clients to support NFS file locking. These daemons must also run on the NFS servers.

lockd

This daemon supports record-locking operations on NFS files. It will send locking requests from the client to the NFS server. On the NFS server, it will start local locking. The daemon is normally started without any options. You can use three options with this command (see the lockd(1M) man page).

The -g graceperiod option selects the number of seconds that the clients have to reclaim locks after a server reboot. During this time, the NFS server will only process reclaims of old locks. All other requests for service must wait until the grace period is over. This option affects the NFS server-side response, so can be can be changed only on an NFS server. The default value for graceperiod is 45 seconds. Reducing this value means that NFS clients can resume operation more quickly after a server reboot, but it increases the chances that a client might not be able to recover all its locks.

The -t timeout option selects the number of seconds to wait before retransmitting a lock request to the remote server. This option affects the NFS client-side service. The default value for timeout is 15 seconds. Decreasing the timeout value can improve response time for NFS clients on a noisy network, but it can cause additional server load by increasing the frequency of lock requests.

The nthreads option specifies the maximum number of concurrent threads that the server will handle per connection. Base the value for nthreads should on the load expected on the NFS server. The default value is 20. Since each NFS client using TCP uses a single connection with the NFS server, each TCP client will be granted the ability to use up to 20 concurrent threads on the server. All NFS clients using UDP will share a single connection with the NFS server. Under these conditions it might be necessary to increase the number of threads available for the UDP connection. A minimum calculation would be to allow for two threads for each UDP client, but this is specific to the workload on the client, so two threads per client might not be sufficient. The disadvantage to using more threads is that when the threads are used, more memory will be used on the NFS server, but if the threads are never used increasing nthreads will have no effect.

mountd

This is a remote procedure call (RPC) server that handles file system mount requests from remote systems and provides access control. It checks /etc/dfs/sharetab to determine which file systems are available for remote mounting and which systems are allowed to do the remote mounting. Two options might be used with this command (see the mountd(1M) man page).

The -v option runs the command in verbose mode. Each time an NFS server determines the access a client should get, a message will be printed on the console. The information generated can be useful when trying to determine why a client can not access a file system.

The -r option rejects all future mount requests from clients. This does not affect clients that already have a file system mounted.

nfsd

This daemon handles other client file-system requests. You can use several options with this command. See the nfsd(1M) man page for a complete listing.

The -l option sets the connection queue length for the NFS/TCP over connection-oriented transports. The default value is 32 entries.

The -c #_conn option selects the maximum number of connections per connection-oriented transport. The default value for #_conn is unlimited.

The nservers option is the maximum number of concurrent requests that a server can handle. The default value for nservers is 1, but the startup scripts select 16.

Unlike older versions of this daemon, nfsd does not spawn multiple copies to handle concurrent requests. Checking the process table with ps only shows one copy of the daemon running.

statd

This daemon works with lockd to provide crash and recovery functions for the lock manager. It keeps track of the clients that hold locks on an NFS server. If a server crashes, upon rebooting statd on the server contacts statd on the client. The client statd can then attempt to reclaim any locks on the server. The client statd also informs the server statd when a client has crashed, so that the client's locks on the server can be cleared. There are no options to select with this daemon. For more information see the statd(1M) man page.

NFS Commands

These commands must be run as root to be fully effective, but requests for information can be made by all users:

clear_locks

This command enables you to remove all file, record, and share locks for an NFS client. You must be root to run this command. From an NFS server you can clear the locks for a specific client and from an NFS client you can clear locks for that client on a specific server. The following example would clear the locks for the NFS client named tulip on the current system.


# clear_locks tulip

Using the -s option enables you to specify which NFS host to clear the locks from. It must be run from the NFS client, which created the locks. In this case, the locks from the client would be removed from the NFS server named bee.


# clear_locks -s bee
Caution - Caution -

This command should only be run when a client crashes and cannot clear its locks. To avoid data corruption problems, do not clear locks for an active client.

mount

With this command, you can attach a named file system, either local or remote, to a specified mount point. For more information, see the mount(1M) man page. Used without arguments, mount displays a list of file systems that are currently mounted on your computer.

Many types of file systems are included in the standard Solaris installation. For a complete description of all of the file system types, see the System Administration Guide. Each file system type has a specific man page that lists the options to mount that are appropriate for that file system type. The man page for NFS file systems is mount_nfs(1M); for UFS file systems it is mount_ufs(1M); and so forth.

Caution - Caution -

The version of the mount command released in Solaris 2.6 and in future patches will not warn about invalid options. The command will silently ignore any options that can not be interpreted. Make sure to verify all of the options that were used to prevent unexpected behavior.

mount Options for NFS File Systems

The subsequent text lists some of the options that can follow the -o flag when mounting an NFS file system.

bg|fg

These options can be used to select the retry behavior if a mount fails. The -bg option causes the mount attempts to be run in the background. The -fg option causes the mount attempt to be run in the foreground. The default is -fg, which is the best selection for file systems that must be available. It prevents further processing until the mount is complete. -bg is a good selection for file systems that are not critical, because the client will do other processing while waiting for the mount request to complete.

largefiles

This option makes it possible to access files larger than 2 Gbytes on a server running the Solaris 2.6 release. Whether a large file can be accessed can only be controlled on the server, so this option is silently ignored on NFS version 3 mounts. By default, all 2.6 UFS file systems are mounted with -largefiles. For mounts using the NFS version 2 protocol, the -largefiles option causes the mount to fail with an error.

nolargefiles

This option for UFS mounts guarantees that there are and will be no large files on the file system (see the mount_ufs(1M) man page). Because the existence of large files can only be controlled on the NFS server, there is no option for -nolargefiles using NFS mounts. Attempts to NFS mount a file system using this option will be rejected with an error.

rw|ro

The -rw and -ro options indicate whether a file system is to be mounted read-write or read-only. The default is read-write, which is the appropriate option for remote home directories, mail-spooling directories, or other file systems that need to be changed by users. The read-only option is appropriate for directories that should not be changed by users; for example, shared copies of the man pages should not be writable by users.

sec=mode

You can use this option to specify the authentication mechanism to be used during the mount transaction. The value for mode can be one of the values shown in Table 3-2. The modes are also defined in /etc/nfssec.conf.

Table 3-2 NFS Security Modes

Mode 

Authentication Service Selected 

krb4 

Kerberos Version 4 

none 

No authentication 

dh 

Diffie-Hellman (DH) authentication 

sys 

Standard UNIX authentication 

soft|hard

An NFS file system mounted with the soft option returns an error if the server does not respond. The hard option causes the mount to continue to retry until the server responds. The default is hard, which should be used for most file systems. Applications frequently do not check return values from soft-mounted file systems, which can make the application fail or can lead to corrupted files. Even if the application does check, routing problems and other conditions can still confuse the application or lead to file corruption if the soft option is used. In most cases the soft option should not be used. If a file system is mounted using the hard option and becomes unavailable, an application using this file system will hang until the file system becomes available.

Using the mount Command

Both of these commands mount an NFS file system from the server bee read-only: 


# mount -F nfs -r bee:/export/share/man /usr/man


# mount -F nfs -o ro bee:/export/share/man /usr/man

This command uses the -O option to force the man pages from the server bee to be mounted on the local system even if /usr/man has already been mounted on: 


# mount -F nfs -O bee:/export/share/man /usr/man

This command uses client failover:


# mount -F nfs -r bee,wasp:/export/share/man /usr/man
Note -

When used from the command line, the listed servers must support the same version of the NFS protocol. Do not mix version 2 and version 3 servers when running mount from the command line. You can use mixed servers with autofs, in which case the best subset of version 2 or version 3 servers is used.

Use the mount command with no arguments to display file systems mounted on a client. 


% mount
/ on /dev/dsk/c0t3d0s0 read/write/setuid on Tues Jan 24 13:20:47 1995
/usr on /dev/dsk/c0t3d0s6 read/write/setuid on Tues Jan 24 13:20:47 1995
/proc on /proc read/write/setuid on Tues Jan 24 13:20:47 1995
/dev/fd on fd read/write/setuid on Tues Jan 24 13:20:47 1995
/tmp on swap read/write on Tues Jan 24 13:20:51 1995
/opt on /dev/dsk/c0t3d0s5 setuid/read/write on Tues Jan 24 13:20:51 1995
/home/kathys on bee:/export/home/bee7/kathys              
  intr/noquota/nosuid/remote on Tues Jan 24 13:22:13 1995

umount

This command enables you to remove a remote file system that is currently mounted. The umount command supports the -V option to allow for testing. You might also use the -a option to umount several file systems at one time. If mount_points are included with the -a option, then those file systems are unmounted. If no mount points are included, then an attempt is made to unmount all file systems listed in /etc/mnttab, except for the "required" file systems, such as /, /usr, /var, /proc, /dev/fd, and /tmp.

Because the file system is already mounted and should have an entry in /etc/mnttab you do not need to include a flag for the file system type.

The command will not succeed if the file system is in use. For instance, if a user has used cd to get access to a file system, the file system will be busy until the working directory is changed. The umount command may hang temporarily if the NFS server is unreachable.

Using the umount Command

This example unmounts a file system mounted on /usr/man: 


# umount /usr/man

This example displays the results of running umount -a -V: 


# umount -a -V
umount /home/kathys
umount /opt
umount /home
umount /net

Note that this command will not actually unmount the file systems.

mountall

Use this command to mount all file systems or a specific group of file systems listed in a file system table. The command provides a way to select the file system type to be accessed with the -F FSType option, to select all the remote file systems listed in a file system table with the -r option, and to select all the local file systems with the -l option. Because all file systems labeled as NFS file system type are remote file systems, some of these options are redundant. For more information, see the mountall(1M) man page.

Using the mountall Command

These two examples are equivalent: 


# mountall -F nfs


# mountall -F nfs -r

umountall

Use this command to unmount a group of file systems. The -k option indicates that the fuser -k mount_point command should be used to kill any processes associated with the mount_point. The -s option indicates that unmount is not to be performed in parallel. -l specifies that only local file systems are to be used, and -r specifies that only remote file systems are to be used. The -h host option indicates that all file systems from the named host should be unmounted. You cannot combine the -h option with -l or -r.

Using the umountall Command

This command unmounts all file systems that are mounted from remote hosts: 


# umountall -r

This command unmounts all file systems currently mounted from the server bee: 


# umountall -h bee

share

With this command, you can make a local file system on an NFS server available for mounting. You can also use the share command to display a list of the file systems on your system that are currently shared. The NFS server must be running for the share command to work. The NFS server software is started automatically during boot if there is an entry in /etc/dfs/dfstab. The command will not report an error if the NFS server software is not running, so you must check this yourself.

The objects that can be shared include any directory tree, but each file system hierarchy is limited by the disk slice or partition that the file system is located on. For instance, sharing the root (/) file system would not also share /usr, unless they are on the same disk partition or slice. Normal installation places root on slice 0 and /usr on slice 6. Also, sharing /usr would not share any other local disk partitions that are mounted on subdirectories of /usr.

A file system cannot be shared that is part of a larger file system that is already shared. For example, if /usr and /usr/local are on one disk slice, then /usr can be shared or /usr/local can be shared, but if both need to be shared with different share options, then /usr/local will need to be moved to a separate disk slice.

Note -

You can gain access to a file system that is shared read-only through the file handle of a file system that is shared read-write if the two file systems are on the same disk slice. It is more secure to place those file systems that need to be read-write on a separate partition or disk slice than the file systems that you need to share read-only.

share Options

Some of the options that you can include with the -o flag are:

rw|ro

The pathname file system is shared read-write or read-only to all clients.

rw=accesslist

The file system is shared read-write to the listed clients only. All other requests are denied. The list of clients defined in accesslist has been expanded for the Solaris 2.6 release. See "Setting Access Lists With the share Command" for more information. You can use this option to override an -ro option.

The options that you can use with NFS file systems only include:

aclok

This option enables an NFS server supporting the NFS version 2 protocol to be configured to do access control for NFS version 2 clients (running SunOS 2.4 or earlier releases). Without this option all clients are given minimal access. With this option the clients have maximal access. For instance, on file systems shared with the -aclok option, if anyone has read permissions, then everyone does. However, without this option, it is possible to deny access to a client who should have access permissions. Whether it is preferred to permit too much access or to permit too little, will depend on the security systems already in place. See the System Administration Guide for more information about access control lists (ACLs).

Note -

To take advantage of ACLs, it is best to have clients and servers run software that supports the NFS version 3 and NFS_ACL protocols. If the software only supports the NFS version 3 protocol, then clients will get correct access, but will not be able to manipulate the ACLs. If the software supports the NFS_ACL protocol, then the clients will get correct access and the capability to manipulate the ACLs. Starting with release 2.5, the Solaris system supports both protocols.

anon=uid

You use uid to select the user ID of unauthenticated users. If you set uid to -1, the server denies access to unauthenticated users. You can grant root access by setting anon=0, but this will allow unauthenticated users to have root access, so use the root option instead.

index=filename

You can use the -index=filename option to force the loading of a HyperText Markup Language (HTML) file instead of displaying a listing of the directory when a user accesses an NFS URL. This option will mimic the action of current browsers if an index.html file is found in the directory that the HTTP URL is accessing. This is the equivalent of setting the DirectoryIndex option for httpd. For instance, if the dfstab file entry looks like:


share -F nfs -o ro,public,index=index.html /export/web

these URLs will display the same information:


nfs://<server>/<dir>
nfs://<server>/<dir>/index.html
nfs://<server>//export/web/<dir>
nfs://<server>//export/web/<dir>/index.html
http://<server>/<dir>
http://<server>/<dir>/index.html
nosuid

This option signals that all attempts to enable the setuid or setgid mode should be ignored. NFS clients will not be able to create files with the setuid or setgid bits on.

public

The -public option has been added to the share command to enable WebNFS browsing. Only one file system on a server may be shared with this option.

root=accesslist

The server gives root access to the hosts in the list. By default, the server does not give root access to any remote hosts. If the selected security mode is anything other than -sec=sys, then you can only include client host names in the accesslist. The list of clients defined in accesslist has been expanded for the Solaris 2.6 release. See "Setting Access Lists With the share Command" for more information.

Caution - Caution -

Granting root access to other hosts has far-reaching security implications; use the root= option with extreme caution.

sec=mode[:mode]

mode selects the security modes that are needed to get access to the file system. By default, the security mode is UNIX authentication. You can specify multiple modes but only use each security mode once per command line. Each -mode option applies to any subsequent -rw, -ro, -rw=, -ro=, -root=, and -window= options, until another -mode is encountered. Using -sec=none maps all users to user nobody.

window=value

value selects the maximum life time in seconds of a credential on the NFS server. The default value is 30000 seconds or 8.3 hours.

Setting Access Lists With the share Command

In OS releases prior to 2.6, the accesslist included with either the -ro=, -rw=, or -root= option of the share command were restricted to a list of host names or netgroup names. In the Solaris 2.6 release, the access list can also include a domain name, a subnet number, or an entry to deny access. These extensions should make it easier to control file access control on a single server, without having to change the name space or maintain long lists of clients.

This command provides read-only access for most systems but allows read-write access for rose and lilac: 


# share -F nfs -o ro,rw=rose:lilac /usr/src

In the next example, read-only access is assigned to any host in the eng netgroup. The client rose is specifically given read-write access. 


# share -F nfs -o ro=eng,rw=rose /usr/src
Note -

You cannot specify both rw and ro without arguments. If no read-write option is specified, the default is read-write for all clients.

To share one file system with multiple clients, you must enter all options on the same line, as multiple invocations of the share command on the same object "remember" only the last command run. This command enables read-write access to three client systems, but only rose and tulip are given access to the file system as root.


# share -F nfs -o rw=rose:lilac:tulip,root=rose:tulip /usr/src

When sharing a file system using multiple authentication mechanisms, make sure to include the -ro, -ro=, -rw, -rw=, -root, and -window options after the correct security modes. In this example, UNIX authentication is selected for all hosts in the netgroup named eng. These hosts can only mount the file system in read-only mode. The hosts tulip and lilac will be able to mount the file system read-write if they use Diffie-Hellman authentication. With these options, tulip and lilac will be able to mount the file system read-only even if they are not using DH authentication, if the host names are listed in the eng netgroup. 


# share -F nfs -o sec=dh,rw=tulip:lilac,sec=sys,ro=eng /usr/src

Even though UNIX authentication is the default security mode, it is not included if the -sec option is used, so it is important to include a -sec=sys option if UNIX authentication is to be used with any other authentication mechanism.

You can use a DNS domain name in the access list by preceding the actual domain name with a dot. The dot indicates that the string following it is a domain name, not a fully qualified host name. The following entry will allow mount access to all hosts in the eng.sun.com domain:


# share -F nfs -o ro=.:.eng.sun.com /export/share/man

In this example, the single "." matches all hosts that are matched through the NIS or NIS+ name spaces. The results returned from these name services do not include the domain name. The ".eng.sun.com" entry matches all hosts that use DNS for name space resolution. DNS always returns a fully qualified host name, so the longer entry is required if you use a combination of DNS and the other name spaces.

You can use a subnet number in an access list by preceding the actual network number or the network name with "@". This differentiates the network name from a netgroup or a fully qualified host name. You must identify the subnet in either /etc/networks or in a NIS or NIS+ name space. The following entries will have the same effect if the 129.144 subnet has been identified as the eng network:


# share -F nfs -o ro=@eng /export/share/man
# share -F nfs -o ro=@129.144 /export/share/man
# share -F nfs -o ro=@129.144.0.0 /export/share/man

The last two entries show that it is not necessary to include the full network address.

If the network prefix is not byte aligned, as with Classless Inter-Domain Routing (CIDR), the mask length can be explicitly specified on the command line. The mask length is defined by following either the network name or the network number with a slash and the number of significant bits in the prefix of the address. For example:


# share -f nfs -o ro=@eng/17 /export/share/man
# share -F nfs -o ro=@129.144.132/17 /export/share/man

In these examples, the "/17" indicates that the first 17 bits in the address are to be used as the mask. For additional information on CIDR, look up RFC 1519.

You can also select negative access by placing a "-" before the entry. Note that because the entries are read from left to right, you must place the negative access entries before the entry they are to apply to:


# share -F nfs -o ro=-rose:.eng.sun.com /export/share/man

This example would allow access to any hosts in the eng.sun.com domain except the host named rose.

unshare

This command allows you to make a previously available file system unavailable for mounting by clients. You can use the unshare command to unshare any file system--whether the file system was shared explicitly with the share command or automatically through /etc/dfs/dfstab. If you use the unshare command to unshare a file system that you shared through the dfstab file, remember that it will be shared again when you exit and re-enter run level 3. You must remove the entry for this file system from the dfstab file if the change is to continue.

When you unshare an NFS file system, access from clients with existing mounts is inhibited. The file system might still be mounted on the client, but the files will not be accessible.

Using the unshare Command

This command unshares a specific file system:


# unshare /usr/src

shareall

This command allows for multiple file systems to be shared. When used with no options, the command shares all entries in /etc/dfs/dfstab. You can include a file name to specify the name of a file that lists share command lines. If you do not include a file name, /etc/dfs/dfstab is checked. If you use a "-" to replace the file name, then you can type share commands from standard input.

Using the shareall Command

This command shares all file systems listed in a local file: 


# shareall /etc/dfs/special_dfstab

unshareall

This command makes all currently shared resources unavailable. The -F FSType option selects a list of file system types defined in /etc/dfs/fstypes. This flag enables you to choose only certain types of file systems to be unshared. The default file system type is defined in /etc/dfs/fstypes. To choose specific file systems, use the unshare command.

Using the unshareall Command

This example should unshare all NFS type file systems:


# unshareall -F nfs

showmount

This command displays all the clients that have remotely mounted file systems that are shared from an NFS server, or only the file systems that are mounted by clients, or the shared file systems with the client access information. The command syntax is:

showmount [ -ade ] [ hostname ]

where -a prints a list all the remote mounts (each entry includes the client name and the directory), -d prints a list of the directories that are remotely mounted by clients, -e prints a list of the files shared (or exported), and hostname selects the NFS server to gather the information from. If hostname is not specified then the local host is queried.

Using the showmount Command

This command lists all clients and the directory that they have mounted.


# showmount -a bee
lilac:/export/share/man
lilac:/usr/src
rose:/usr/src
tulip:/export/share/man

This command lists the directories that have been mounted.


# showmount -d bee
/export/share/man
/usr/src

This command lists file systems that have been shared.


# showmount -e bee
/usr/src								(everyone)
/export/share/man					eng

setmnt

This command creates an /etc/mnttab table. The mount and umount commands consult the table. Generally, there is no reason to run this command manually; it is run automatically when a system is booted.

Other Useful Commands

These commands can be useful when troubleshooting NFS problems.

nfsstat

You can use this command to gather statistical information about NFS and RPC connections. The syntax of the command is:

nfsstat [ -cmnrsz ]

where -c displays client-side information, -m displays statistics for each NFS-mounted file system, -n specifies that NFS information is to be displayed (both client and server side), -r displays RPC statistics, -s displays the server-side information, and -z specifies that the statistics should be set to zero. If no options are supplied on the command line, the -cnrs options are used.

Gathering server-side statistics can be important for debugging problems when new software or hardware is added to the computing environment. Running this command at least once a week, and storing the numbers provides a good history of previous performance.

Using the nfsstat Command


# nfsstat -s

Server rpc:
Connection oriented:
calls      badcalls   nullrecv   badlen      xdrcall    dupchecks  dupreqs
11420263   0          0          0           0          1428274    19
Connectionless:
calls      badcalls   nullrecv   badlen      xdrcall    dupchecks  dupreqs
14569706   0          0          0           0          953332     1601

Server nfs:
calls      badcalls
24234967   226
Version 2: (13073528 calls)
null       getattr    setattr    root        lookup     readlink   read
138612 1%  1192059 9% 45676 0%   0 0%        9300029 71% 9872 0%   1319897 10%
wrcache    write      create     remove      rename     link       symlink
0 0%       805444 6%  43417 0%   44951 0%    3831 0%    4758 0%    1490 0%
mkdir      rmdir      readdir    statfs
2235 0%    1518 0%    51897 0%   107842 0%
Version 3: (11114810 calls)
null       getattr    setattr    lookup      access     readlink   read
141059 1%  3911728 35% 181185 1% 3395029 30% 1097018 9% 4777 0%   960503 8%
write      create     mkdir      symlink     mknod      remove     rmdir
763996 6%  159257 1%  3997 0%    10532 0%    26 0%      164698 1%  2251 0%
rename     link       readdir    readdirplus fsstat     fsinfo     pathconf
53303 0%   9500 0%    62022 0%   79512 0%    3442 0%    34275 0%   3023 0%
commit    
73677 0%  

Server nfs_acl:
Version 2: (1579 calls)
null       getacl     setacl     getattr     access    
0 0%       3 0%       0 0%       1000 63%    576 36%   
Version 3: (45318 calls)
null       getacl     setacl    
0 0%       45318 100% 0 0%

This is an example of NFS server statistics. The first five lines deal with RPC and the remaining lines report NFS activities. In both sets of statistics knowing the average number of badcalls or calls and the number of calls per week, can help identify when something is going wrong. The badcalls value reports the number of bad messages from a client and can point out network hardware problems.

Some of the connections generate write activity on the disks. A sudden increase in these statistics could indicate trouble and should be investigated. For NFS version 2 statistics, the connections to note to are: setattr, write, create, remove, rename, link, symlink, mkdir, and rmdir. For NFS version 3 statistics the value to watch is commit. If the commit level is high in one NFS server as compared to another almost identical one, check that the NFS clients have enough memory. The number of commit operations on the server go up when clients do not have resources available.

pstack

This command displays a stack trace for each process. It must be run by root. You can use it to determine where a process is hung. The only option allowed with this command is the PID of the process that you want to check (see the proc(1) man page).

The example below is checking the nfsd process that is running.


# /usr/proc/bin/pstack 243
243:    /usr/lib/nfs/nfsd -a 16
 ef675c04 poll     (24d50, 2, ffffffff)
 000115dc ???????? (24000, 132c4, 276d8, 1329c, 276d8, 0)
 00011390 main     (3, efffff14, 0, 0, ffffffff, 400) + 3c8
 00010fb0 _start   (0, 0, 0, 0, 0, 0) + 5c

It shows that the process is waiting for a new connection request. This is a normal response. If the stack shows that the process is still in poll after a request is made, it is possible that the process is hung. Follow the instructions in "How to Restart NFS Services" to fix this problem. Review the instructions in "NFS Troubleshooting Procedures" to fully verify that your problem is a hung program.

rpcinfo

This command generates information about the RPC service running on a system. You can also use it to change the RPC service. There are many options available with this command (see the rpcinfo(1M) man page). This is a shortened synopsis for some of the options that you can use with the command:

rpcinfo [ -m | -s ] [ hostname ]

rpcinfo [ -t | -u ] [ hostname ] [ progname ]

where -m displays a table of statistics of the rpcbind operations, -s displays a concise list of all registered RPC programs, -t displays the RPC programs that use TCP, -u displays the RPC programs that use UDP, hostname selects the host name of the server you need information from, and progname selects the RPC program to gather information about. If no value is given for hostname, then the local host name is used. You can substitute the RPC program number for progname, but many users will remember the name and not the number. You can use the -p option in place of the -s option on those systems that do not run the NFS version 3 software.

The data generated by this command can include:

Using the rpcinfo Command

This example gathers information on the RPC services running on a server. The text generated by the command is filtered by the sort command to make it more readable. Several lines listing RPC services have been deleted from the example.


% rpcinfo -s bee |sort -n
   program version(s) netid(s)                         service     owner
    100000  2,3,4     udp,tcp,ticlts,ticotsord,ticots  portmapper  superuser
    100001  4,3,2     ticlts,udp                       rstatd      superuser
    100002  3,2       ticots,ticotsord,tcp,ticlts,udp  rusersd     superuser
    100003  3,2       tcp,udp                          nfs         superuser
    100005  3,2,1     ticots,ticotsord,tcp,ticlts,udp  mountd      superuser
    100008  1         ticlts,udp                       walld       superuser
    100011  1         ticlts,udp                       rquotad     superuser
    100012  1         ticlts,udp                       sprayd      superuser
    100021  4,3,2,1   ticots,ticotsord,ticlts,tcp,udp  nlockmgr    superuser
    100024  1         ticots,ticotsord,ticlts,tcp,udp  status      superuser
    100026  1         ticots,ticotsord,ticlts,tcp,udp  bootparam   superuser
    100029  2,1       ticots,ticotsord,ticlts          keyserv     superuser
    100068  4,3,2     tcp,udp                          cmsd        superuser
    100078  4         ticots,ticotsord,ticlts          kerbd       superuser
    100083  1         tcp,udp                          -           superuser
    100087  11        udp                              adm_agent   superuser
    100088  1         udp,tcp                          -           superuser
    100089  1         tcp                              -           superuser
    100099  1         ticots,ticotsord,ticlts          pld         superuser
    100101  10        tcp,udp                          event       superuser
    100104  10        udp                              sync        superuser
    100105  10        udp                              diskinfo    superuser
    100107  10        udp                              hostperf    superuser
    100109  10        udp                              activity    superuser
	.
	.
    100227  3,2       tcp,udp                          -           superuser
    100301  1         ticlts                           niscachemgr superuser
    390100  3         udp                              -           superuser
1342177279  1,2       tcp                              -           14072

This example shows how to gather information about a particular RPC service using a particular transport on a server.


% rpcinfo -t bee mountd
program 100005 version 1 ready and waiting
program 100005 version 2 ready and waiting
program 100005 version 3 ready and waiting
% rpcinfo -u bee nfs
program 100003 version 2 ready and waiting
program 100003 version 3 ready and waiting

The first example checks the mountd service running over TCP. The second example checks the NFS service running over UDP.

snoop

This command is often used to watch for packets on the network. It must be run as root. It is a good way to make sure that the network hardware is functioning on both the client and the server. Many options are available (see the snoop(1M) man page). A shortened synopsis of the command is given below:

snoop [ -d device ] [ -o filename ] [ host hostname ]

where -d device specifies the local network interface, -o filename stores all the captured packets into the named file, and hostname indicates to display only packets going to and from a specific host.

The -d device option is useful on those servers that have multiple network interfaces. You can use many other expressions besides setting the host. A combination of command expressions with grep can often generate data that is specific enough to be useful.

When troubleshooting make sure that packets are going to and from the host that you expect them to. Also, look for error messages. Saving the packets to a file can make it much easier to review the data.

truss

You can use this command to see if a process is hung. It must be run by root. You can use many options with this command (see the truss(1) man page). A shortened syntax of the command is:

truss [ -t syscall ] -p pid

where -t syscall selects system calls to trace, and -p pid indicates the PID of the process to be traced. The syscall may be a comma-separated list of system calls to be traced. Also, starting syscall with a ! selects to exclude the system calls from the trace.

This example shows that the process is waiting for another connection request from a new client. 


# /usr/bin/truss -p 243
poll(0x00024D50, 2, -1)         (sleeping...)

This is a normal response. If the response does not change after a new connection request has been made, the process could be hung. Follow the instructions in "How to Restart NFS Services" to fix the hung program. Review the instructions in "NFS Troubleshooting Procedures" to fully verify that your problem is a hung program.

How It All Works Together

The following sections describe some of the complex functions of the NFS software.

Version 2 and Version 3 Negotiation

Because NFS servers might be supporting clients that are not using the NFS version 3 software, part of the initiation procedure includes negotiation of the protocol level. If both the client and the server can support version 3, then that version will be used. If either the client or the server can only support version 2, then that version will be selected.

You can override the values determined by the negotiation by using the -vers option to the mount command (see the mount_nfs(1M) man page). Under most circumstances, you should not have to specify the version level, as the best one will be selected by default.

UDP and TCP Negotiation

During initiation, the transport protocol is also negotiated. By default, the first connection-oriented transport supported on both the client and the server is selected. If this does not succeed, then the first available connectionless transport protocol is used. The transport protocols supported on a system are listed in /etc/netconfig. TCP is the connection-oriented transport protocol supported by the Solaris 2.6 release. UDP is the connectionless transport protocol.

When both the NFS protocol version and the transport protocol are determined by negotiation, the NFS protocol version is given precedence over the transport protocol. The NFS version 3 protocol using UDP will be given higher precedence than the NFS version 2 protocol using TCP. You can manually select both the NFS protocol version and the transport protocol with the mount command (see the mount_nfs(1M) man page). Under most conditions, it is better to allow the negotiation to select the best options.

File Transfer Size Negotiation

The file transfer size establishes the size of the buffers that are used when transferring data between the client and the server. In general, larger transfer sizes are better. The NFS version 3 protocol has an unlimited transfer size, but the Solaris 2.6 release bids a default buffer size of 32 Kbytes. The client can bid a smaller transfer size at mount time if needed, but under most conditions this is not necessary.

The transfer size is not negotiated with systems using the NFS version 2 protocol. Under this condition the maximum transfer size is set to 8 Kbytes.

You can use the -rsize and -wsize options to manually set the transfer size with the mount command. You might need to reduce the transfer size for some PC clients. Also, you can increase the transfer size if the NFS server is configured to use larger transfer sizes.

Client-Side Failover

Using client-side failover, an NFS client can switch to another server if the server supporting a replicated file system becomes unavailable. The file system can become unavailable if the server it is connected to crashes, if the server is overloaded or if there is a network fault. The failover, under these conditions, will normally be transparent to the user. Once established the failover can occur at any time without disrupting the processes running on the client.

Failover requires that the file system be mounted read-only. The file systems must be identical for the failover to occur successfully. See "What Is a Replicated File System?" for a description of what makes a file system identical. A static file system or one that is not changed often is the best candidate for failover.

You can not use file systems that are mounted using cachefs with failover. Extra information is stored for each cachefs file system. This information can not be updated during failover, so only one of these two features may be used when mounting a file system.

The number of replicas that need to be established for each file system depends on many factors. In general, it is better to have a couple of servers, each supporting multiple subnets rather than have a unique server on each subnet. The process requires checking of each server in the list, so the more servers that are listed the slower each mount will be.

Failover Terminology

To fully comprehend the process, two terms need to be understood.

What Is a Replicated File System?

For the purposes of failover, a file system may be called a replica when each file is the same size and has the same vnode type as the original file system. Permissions, creation dates, and other file attributes are not considered. If the file size or vnode types are different, then the remap will fail and the process will hang until the old server becomes available.

You can maintain a replicated file system using rdist, cpio, or other file transfer mechanisms. Because updating the replicated file systems will cause inconsistency, follow these suggestions for best results:

Failover and NFS Locking

Some software packages require read locks on files. To prevent these products from breaking, read locks on read-only file systems are allowed, but are visible to the client side only. The locks will persist through a remap because the server doesn't "know" about them. Because the files should not be changing, you do not need to lock the file on the server side.

Large Files

The Solaris 2.6 release supports files that are over 2 Gbytes. By default, UFS file systems are mounted with the -largefiles option to support the new functionality. Previous releases are not able to handle files of this size. See "How to Disable Large Files on an NFS Server" for instructions.

No changes need to occur on a Solaris 2.6 NFS client to be able to access a large file, if the file system on the server is mounted with the -largefiles option. However, not all 2.6 commands will be able to handle these large files. See largefile(5) for a list of the commands that can handle the large files. Clients that cannot support the NFS version 3 protocol with the large file extensions will be unable to access any large files. Although clients running the Solaris 2.5 release can use the NFS version 3 protocol, large file support was not included in that release.

How the WebNFS Service Works

The WebNFS service makes files in a directory available to clients using a public file handle. A file handle is simply an address generated by the kernel that identifies a file for NFS clients. The public file handle has a predefined value, so there is no need for the server to generate a file handle for the client. The ability to use this predefined file handle reduces network traffic by eliminating the MOUNT protocol and should increase response time for the clients.

By default the public file handle on an NFS server is established on the root file system. This default will provide WebNFS access to any clients that already have mount privileges on the server. You can change the public file handle to point to any file system by using the share command.

When the client has the file handle for the file system, a LOOKUP is run to determine the file handle for the file to be accessed. The NFS protocol allows the evaluation of only one path name component at a time. Each additional level of directory hierarchy requires another LOOKUP. A WebNFS server can evaluate an entire path name with a single transaction, called multicomponent lookup, when the LOOKUP is relative to the public file handle. With multicomponent lookup, the WebNFS server is able to deliver the file handle to the desired file without having to exchange the file handles for each directory level in the path name.

In addition, an NFS client can initiate concurrent downloads over a single TCP connection, which provides quick access without the additional load on the server caused by setting up multiple connections. Although Web browser applications support concurrent downloading of multiple files, each file has its own connection. By using one connection, the WebNFS software reduces the overhead on the server.

If the final component in the path name is a symbolic link to another file system, the client can access the file if the client already has access through normal NFS activities.

Normally, an NFS URL will be evaluated relative to the public file handle. The evaluation can be changed to be relative to the server's root file system by adding an additional slash to the beginning of the path. In this example, these two NFS URLs would be equivalent if the public file handle has been established on the /export/ftp file system.


nfs://server/junk
nfs://server//export/ftp/junk

WebNFS Limitations With Web Browser Use

There are several functions that a Web site using HTTP can provide that are not supported by the WebNFS software. These differences are caused by the fact that the NFS server will only send the file, so any special processing must be done on the client. If you need to have one Web site that is configured for both WebNFS and HTTP access, then consider the following issues:

The Secure NFS System

The NFS environment is a powerful and convenient way to share file systems on a network of different computer architectures and operating systems. However, the same features that make sharing file systems through NFS operation convenient also pose some security problems. Historically, most NFS implementations have used UNIX (or AUTH_SYS) authentication, but stronger authentication methods such as AUTH_DH have also been available. When using UNIX authentication, an NFS server authenticates a file request by authenticating the computer making the request, but not the user, so a client user can run su and impersonate the owner of a file. If DH authentication is used, the NFS server will authenticate the user, making this sort of impersonation much harder.

With root access and knowledge of network programming, anyone can introduce arbitrary data into the network and extract any data from the network. The most dangerous attacks are those involving the introduction of data, such as impersonating a user by generating the right packets or recording "conversations" and replaying them later. These attacks affect data integrity. Attacks involving passive eavesdropping--merely listening to network traffic without impersonating anybody--are not as dangerous, as data integrity is not compromised. Users can protect the privacy of sensitive information by encrypting data that goes over the network.

A common approach to network security problems is to leave the solution to each application. A better approach is to implement a standard authentication system at a level that covers all applications.

The Solaris operating system includes an authentication system at the level of remote procedure call (RPC)--the mechanism on which NFS operation is built. This system, known as Secure RPC, greatly improves the security of network environments and provides additional security to services such as the NFS system. When the NFS system uses the facilities provided by Secure RPC, it is known as a Secure NFS system.

Secure RPC

Secure RPC is fundamental to the Secure NFS system. The goal of Secure RPC is to build a system at least as secure as a time-sharing system (one in which all users share a single computer). A time-sharing system authenticates a user through a login password. With data encryption standard (DES) authentication, the same is true. Users can log in on any remote computer just as they can on a local terminal, and their login passwords are their passports to network security. In a time-sharing environment, the system administrator has an ethical obligation not to change a password in order to impersonate someone. In Secure RPC, the network administrator is trusted not to alter entries in a database that stores public keys.

You need to be familiar with two terms to understand an RPC authentication system: credentials and verifiers. Using ID badges as an example, the credential is what identifies a person: a name, address, birthday, and so on. The verifier is the photo attached to the badge: you can be sure the badge has not been stolen by checking the photo on the badge against the person carrying it. In RPC, the client process sends both a credential and a verifier to the server with each RPC request. The server sends back only a verifier because the client already "knows" the server's credentials.

RPC's authentication is open ended, which means that a variety of authentication systems may be plugged into it. Currently, there are three systems: UNIX, DH, and KERB (for Kerberos Version 4).

When UNIX authentication is used by a network service, the credentials contain the client's host name, UID, GID, and group-access list, but the verifier contains nothing. Because there is no verifier, a superuser could falsify appropriate credentials, using commands such as su. Another problem with UNIX authentication is that it assumes all computers on a network are UNIX computers. UNIX authentication breaks down when applied to other operating systems in a heterogeneous network.

To overcome the problems of UNIX authentication, Secure RPC uses either DH authentication or KERB authentication.

DH Authentication

DH authentication uses the data encryption standard (DES) and Diffie-Hellman public-key cryptography to authenticate both users and computers in the network. DES is a standard encryption mechanism; Diffie-Hellman public-key cryptography is a cipher system that involves two keys: one public and one secret. The public and secret keys are stored in the name space. NIS stores the keys in the publickey map, and NIS+ stores the keys in the cred table. These maps contain the public key and secret key for all potential users. See the System Administration Guide for more information on how to set up the maps and tables.

The security of DH authentication is based on a sender's ability to encrypt the current time, which the receiver can then decrypt and check against its own clock. The time stamp is encrypted with DES. There are two requirements for this scheme to work:

If a network runs a time-synchronization program, then the time on the client and the server is synchronized automatically. If a time synchronization program is not available, time stamps can be computed using the server's time instead of the network time. The client asks the server for the time before starting the RPC session, then computes the time difference between its own clock and the server's. This difference is used to offset the client's clock when computing time stamps. If the client and server clocks get out of synchronization to the point where the server begins to reject the client's requests, the DH authentication system on the client resynchronizes with the server.

The client and server arrive at the same encryption key by generating a random conversation key, also known as the session key, and then using public-key cryptography to deduce a common key. The common key is a key that only the client and server are capable of deducing. The conversation key is used to encrypt and decrypt the client's time stamp; the common key is used to encrypt and decrypt the conversation key.

KERB Authentication

Kerberos is an authentication system developed at MIT. Encryption in Kerberos is based on DES.

Kerberos works by authenticating the user's login password. A user types the kinit command, which obtains a ticket that is valid for the time of the session (or eight hours, the default session time) from the authentication server. When the user logs out, the ticket may be destroyed using the kdestroy command.

The Kerberos server software is available from MIT Project Athena, and is not part of the SunOS software. SunOS software provides

See the System Administration Guide for more details.

Using Secure RPC With NFS

Be aware of the following points if you plan to use Secure RPC: