Go to main content

ONC+ RPC Developer's Guide

Exit Print View

Updated: March 2019

Authentication Using RPCSEC_GSS

A determined snoop can overcome the authentication flavors mentioned previously – AUTH_SYS and AUTH_DES. For this reason a new networking layer, the Generic Security Standard API, or GSS-API, was added, which RPC programmers can use. The GSS-API framework offers two extra services beyond authentication: integrity and privacy.

  • Integrity – With the integrity service, the GSS-API uses the underlying mechanism to authenticate messages exchanged between programs. Cryptographic checksums establish:

    • The identity of the data originator to the recipient

    • The identity of the recipient to the originator if mutual authentication is requested

    • The authenticity of the transmitted data itself

  • Privacy – The privacy service includes the integrity service. In addition, the transmitted data is also encrypted to protect it from any eavesdroppers.

    Because of U.S. export restrictions, the privacy service might not be available to all users.

Note -  Currently, the GSS-API is exposed, and certain GSS-API features are visible through RPCSEC_GSS functions. See the Developer’s Guide to Oracle Solaris 11.3 Security.


The RPCSEC_GSS security flavor enables ONC RPC applications to maximize the features of GSS-API. RPCSEC_GSS sits "on top" of the GSS-API layer as shown in the following figure.

Figure 3  GSS-API and RPCSEC_GSS Security Layers

image:Graphic shows RPSEC_GSS is between the application and               GSS-API.

Using the programming interface for RPCSEC_GSS, ONC RPC applications can specify the following information:

  • Mechanism – A security paradigm. Each kind of security mechanism offers a different kind of data protection, as well as one or more levels of data protection. You can use any security mechanism supported by the GSS-API (Kerberos V5, RSA public key, and so forth).

  • Security service – Either privacy or integrity or neither. The default is integrity. The service is mechanism independent.

  • QOP – Quality of protection. The QOP specifies the type of cryptographic algorithm to be used to implement privacy or integrity services. Each security mechanism can have one or more QOPs associated with it.

Applications can obtain lists of valid QOPs and mechanisms through functions provided by RPCSEC_GSS. See Miscellaneous Functions. Developers should avoid hard-coding mechanisms and QOPs into their applications, so that the applications do not need to be modified to use new or different mechanisms and QOPs.

Note -  Historically, "security flavor" and "authentication flavor" have had the same meaning. With the introduction of RPCSEC_GSS, "flavor" now has a somewhat different sense. A flavor can now include a service integrity or privacy along with authentication, although currently RPCSEC_GSS is the only flavor that falls into this category.

Using RPCSEC_GSS, ONC RPC applications establish a security context with a peer, exchange data, and destroy the context, just as they do with other flavors. After a context is established, the application can change the QOP and service for each data unit sent.

For more information about RPCSEC_GSS, including RPCSEC_GSS data types, see the rpcsec_gss(3N) man page.


The following table summarizes RPCSEC_GSS commands. It is a general overview of RPCSEC_GSS functions, rather than a specific description of each one. For more information about each function, see its man page, or check the rpcsec_gss(3NSL) man page for an overview, including a list of RPCSEC_GSS data structures.

Table 13  RPCSEC_GSS Functions
Create a security context
CLIENT handle, principal name, mechanism, QOP, service type
AUTH handle
Change QOP, service type for context
Old QOP, service
New QOP, service
Show maximum size for data before security transformation
Maximum data size allowed by transport
Maximum pre-transformation data size
Show maximum size for data before security transformation
Maximum data size allowed by transport
Maximum pre-transformation data size
Set name of principals for server to represent
Principal name, RPC program, version #s
TRUE if successful
Fetch credentials of caller (client)
Pointer to svc_req structure
UNIX credentials, RPCSEC_GSS credentials, cookie
Specify user-written callback function
Pointer to callback function
TRUE if successful
Create RPCSEC_GSS structure for principal names from unique parameters
Mechanism, user name, system name, domain name
RPCSEC_GSS principal name structure
Fetch an error code when an RPCSEC_GSS routine fails
RPCSEC_GSS error number, errno if applicable
Get strings for installed mechanisms
List of valid mechanisms
Get valid QOP strings
Valid QOPs for that mechanism
Get the highest, lowest version numbers of RPCSEC_GSS supported
Highest, lowest versions
Check if a mechanism is installed
TRUE if installed
Convert ASCII mechanism to RPC object identifier
Mechanism (as string)
Mechanism (as OID)
Convert ASCII QOP to integer
QOP (as string)
QOP (as integer)

Creating a Context

    You create contexts with the rpc_gss_seccreate() call. This function takes as its arguments:

  • A client handle returned, for example, by clnt_create()

  • The name of the server principal, for example, nfs@example.com

  • The mechanism (for example, Kerberos V5) for the session

  • The security service type (for example, privacy)

  • The QOP for the session

  • Two GSS-API parameters that can remain opaque for most uses (that is, the programmer can supply NULL values)

This function returns an AUTH authentication handle. The following example shows how rpc_gss_seccreate() might be used to create a context using the Kerberos V5 security mechanism and the integrity service.

Example 59  Using the rpc_gss_seccreate() Function
CLIENT *clnt;                    /* client handle */
char server_host[] = "foo";
char service_name[] = "nfs@eng.example.com";
char mech[] = "kerberos_v5";

clnt = clnt_create(server_host, SERVER_PROG, SERV_VERS, "netpath");
clnt->clnt_auth = rpc_gss_seccreate(clnt, service_name, mech,
                          rpc_gss_svc_integrity, NULL, NULL, NULL);

. . .

    Note the following points about the example:

  • Although the mechanism was declared explicitly for ease of reading, it would be more commonly obtained programmatically with rpc_gss_get_mechanisms() from a table of available mechanisms.

  • The QOP is passed as a NULL, which sets the QOP to this mechanism's default. Otherwise, a valid value could, as with the mechanism, be obtained programmatically with rpc_gss_get_mechanisms(). See the rpc_gss_get_mechanisms(3NSL) man page for more information.

  • The security service type, rpc_gss_svc_integrity, is an enum of the RPCSEC_GSS type rpc_gss_service_t. rpc_gss_service_t has the following format:

    typedef enum {
         rpc_gss_svc_default = 0,
         rpc_gss_svc_none = 1,
         rpc_gss_svc_integrity = 2,
         rpc_gss_svc_privacy = 3
    }  rpc_gss_service_t;

    The default security service maps to integrity, so the programmer could have specified rpc_gss_svc_default and obtained the same result.

For more information, see the rpc_gss_seccreate(3NSL) man page.

Changing Values and Destroying a Context

After a context has been set, the application might need to change QOP and service values for individual data units being transmitted. For example, if you want a program to encrypt a password but not a login name, you can use rpc_gss_set_defaults().

Example 60  rpc_gss_set_defaults() Function
rpc_gss_set_defaults(clnt->clnt_auth, rpc_gss_svc_privacy, qop);

. . .

In this case, the security service is set to privacy. See Creating a Context. qop is a pointer to a string naming the new QOP.

Contexts are destroyed in the usual way, with auth_destroy().

For more information about changing service and QOP, see the rpc_gss_set_defaults(3NSL) man page.

Principal Names

    You need both a client and a server principal name to establish and maintain a security context.

  • A server's principal name is always specified as a NULL-terminated ASCII string of the form service@host. One example is nfs@iteast.example.com.

    When a client creates a security context, it specifies the server principal name in this format. See Creating a Context. Similarly, when a server needs to set the name of a principal it represents, it uses rpc_gss_set_svc_name(). This function takes a principal name in this format as an argument.

  • The principal name of a client, as received by a server, takes the form of an rpc_gss_principal_t structure: a counted, opaque byte string determined by the mechanism being used. This structure is described in the rpcsec_gss(3NSL) man page.

Setting Server Principal Names

A server needs to be told the names of the principals it represents when it starts up. A server can act as more than one principal. rpc_gss_set_svc_name() sets the name of the principals, as shown in the following code example.

Example 61  rpc_gss_set_svc_name() Function
char *principal, *mechanism;
u_int req_time;

principal = "nfs@eng.example.com";
mechanism = "kerberos_v5";
req_time = 10000;		/* time for which credential should be valid */

rpc_gss_set_svc_name(principal, mechanism, req_time, SERV_PROG, SERV_VERS);

Kerberos ignores the req_time parameter. Other authentication systems might use it.

For more information, see the rpc_gss_set_svc_name(3NSL) man page.

Generating Client Principal Names

Servers need to be able to operate on a client's principal name. For example, you might need to compare a client's principal name to an access control list, or look up a UNIX credential for that client, if such a credential exists. Such principal names are kept in the form of a rpc_gss_principal_t structure pointer. See the rpcsec_gss(3NSL) man page for more on rpc_gss_principal_t. If a server is to compare a principal name it has received with the name of a known entity, the server needs to be able to generate a principal name in that form.

The rpc_gss_get_principal_name() call takes as input several parameters that uniquely identify an individual on a network, and generates a principal name as a rpc_gss_principal_t structure pointer, as shown in the following code example.

Example 62  Using the rpc_gss_get_principal_name() Function
rpc_gss_principal_t *principal;
rpc_gss_get_principal_name(principal, mechanism, name, node, domain);
. . .

    The arguments to rpc_gss_get_principal_name() are:

  • principal is a pointer to the rpc_gss_principal_t structure to be set.

  • mechanism is the security mechanism being used. The principal name being generated is mechanism dependent.

  • name is an individual or service name, such as jdoe or nfs, or even the name of a user-defined application.

  • node might be, for example, a UNIX system name.

  • domain might be, for example, a DNS, NIS, or a Kerberos realm.

Each security mechanism requires different identifying parameters. For example, Kerberos V5 requires a user name and, only optionally, qualified node and domain names, which in Kerberos terms are host and realm names.

For more information, see the rpc_gss_get_principal_name(3NSL) man page.

Freeing Principal Names

Use the free() library call to free principal names.

Receiving Credentials at the Server

A server must be able to fetch the credentials of a client. The rpc_gss_getcred() function, shown in Example 63, Getting Credentials, enables the server to retrieve either UNIX credentials or RPCSEC_GSS credentials, or both. The function has two arguments that are set if the function is successful. One is a pointer to an rpc_gss_ucred_t structure, which contains the caller's UNIX credentials, if such exist:

typedef struct {
    uid_t   uid;          /* user ID */
    gid_t   gid;          /* group ID */
    short   gidlen; 
    git_t   *gidlist;     /* list of groups */
} rpc_gss_ucred_t;

The other argument is a pointer to a rpc_gss_raw_cred_t structure, which looks like this:

typedef struct {
	u_int  version;              /*RPCSEC_GS program version *mechanism;
          	char          *qop;
	rpc_gss_principal_t client_principal;  /* client principal name */
	char                   *svc_principal; /*server principal name */
	rpc_gss_service_t	  service;        /* privacy, integrity enum */
} rpc_gss_rawcred_t;

Because rpc_gss_rawcred_t contains both the client and server principal names, rpc_gss_getcred() can return them both. See Generating Client Principal Names for a description of the rpc_gss_principal_t structure and how it is created.

The following example is a simple server-side dispatch procedure, in which the server gets the credentials for the caller. The procedure gets the caller's UNIX credentials and then verifies the user's identity, using the mechanism, QOP, and service type found in the rpc_gss_rcred_t argument.

Example 63  Getting Credentials
static void server_prog(struct svc_req *rqstp, SVCXPRT *xprt)
		rpc_gss_ucred_t *ucred;
		rpc_gss_rawcred_t *rcred;
		if (rqst->rq_proq == NULLPROC) {
			svc_sendreply(xprt, xdr_void, NULL);
		 * authenticate all other requests */
		switch (rqstp->rq_cred.oa_flavor) {
		case RPCSEC_GSS:
			 * get credential information
			rpc_gss_getcred(rqstp, &rcred, &ucred, NULL);
			* verify that the user is allowed to access
			* using received security parameters by
			* peeking into my config file
			if (!authenticate_user(ucred->uid, rcred->mechanism,
				rcred->qop, rcred->service)) {
			break; 	/* allow the user in */
		} /* end switch */
		switch (rqstp->rq_proq) {
		case SERV_PROC1:
			. . .
		/* usual request processing; send response ... */

For more information, see the rpc_gss_getcred(3NSL) man page.

Cookies and RPC

In Example 63, Getting Credentials, the last argument to rpc_gss_getcred() (here, a NULL) is a user-defined cookie, with a value on return of whatever was specified by the server when the context was created. This cookie, a 4-byte value, can be used in any way appropriate for the application. RPC does not interpret the cookie. For example, the cookie can be a pointer or index to a structure that represents the context initiator. Instead of computing this value for every request, the server computes it at context-creation time, saving on request-processing time.

Cookies and Callbacks in RPC

Another opportunity to use cookies is with callbacks. By using the rpc_gss_set_callback() function, a server can specify a user-defined callback so that it knows when a context first gets used. The callback is invoked the first time a context is used for data exchanges, after the context is established for the specified program and version.

The user-defined callback routine takes the following form:

bool_t callback (struct svc_req *req, gss_cred_id_t deleg,
gss_ctx_id_t gss_context rpc_gss_lock_t *
lock void ** cookie);

The second and third arguments, deleg and gss_context, are GSS-API data types and are currently exposed. See the Developer’s Guide to Oracle Solaris 11.3 Security for more information. Note that deleg is the identity of any delegated peer, while gss_context is a pointer to the GSS-API context. This pointer is necessary in case the program needs to perform GSS-API operations on the context, that is, to test for acceptance criteria. You have already seen the cookie argument.

The lock argument is a pointer to a rpc_gss_lock_t structure:

typedef struct {
		bool_t              locked;
		rpc_gss_rawcred_t   *raw_cred;
} rpc_gss_lock_t;

This parameter enables a server to enforce a particular QOP and service for the session. QOP and service are found in the rpc_gss_rawcred_t structure described in Example 63, Getting Credentials. A server should not change the values for service and QOP. When the user-defined callback is invoked, the locked field is set to FALSE. If the server sets locked to TRUE, only requests with QOP and service values that match the QOP and service values in the rpc_gss_rawcred_t structure are accepted.

For more information, see the rpc_gss_set_callback(3NSL) man page.

Maximum Data Size

Two functions, rpc_gss_max_data_length() on the client side, and rpc_gss_svc_max_data_length() on the server side, are useful in determining how large a piece of data can be before it is transformed by security measures and sent "over the wire". A security transformation such as encryption usually changes the size of a piece of transmitted data, most often enlarging it. To make sure that data won't be enlarged past a usable size, these two functions return the maximum pre-transformation size for a given transport.

For more information, see the rpc_gss_max_data_length(3NSL) man page.

Miscellaneous Functions

    You can use several functions for getting information about the installed security system.

  • rpc_gss_get_mechanisms(3NSL)() returns a list of installed security mechanisms.

  • rpc_gss_is_installed(3NSL)() checks if a specified mechanism is installed.

  • rpc_gss_get_mech_info(3NSL)() returns valid QOPs for a given mechanism.

Using these functions gives the programmer latitude in avoiding hard-coding security parameters in applications. (See Figure 13, Table 13, RPCSEC_GSS Functions and the rpcsec_gss(3NSL) man page for a list of all RPCSEC_GSS functions.)

Associated Files

RPCSEC_GSS makes use of certain files to store information.

gsscred Table

When a server retrieves the client credentials associated with a request, the server can get either the client's principal name in the form of a rpc_gss_principal_t structure pointer or local UNIX credentials (UID) for that client. Services such as NFS require a local UNIX credential for access checking, but others might not. Those services can, for example, store the principal name directly in their own access control lists as a rpc_gss_principal_t structure.

Note - The correspondence between a client's network credential (its principal name) and any local UNIX credential is not automatic. The local security administrator must be set up explicitly.

The gsscred file contains both the client's UNIX and network (for example, Kerberos V5) credentials. The network credential is the Hex-ASCII representation of the rpc_gss_principal_t structure. The gsscred file is accessed through XFN. Thus, this table can be implemented over files, NIS, or any future name service supported by XFN. In the XFN hierarchy, this table appears as this_org_unit/service/gsscred. Administrators can maintain the gsscred table with the use of the gsscred utility, which enables adding and deleting of users and mechanisms.

/etc/gss/qop and /etc/gss/mech

For convenience, RPCSEC_GSS uses string literals for representing mechanisms and quality of protection (QOP) parameters. The underlying mechanisms themselves, however, require mechanisms to be represented as object identifiers and QOPs as 32-bit integers. Additionally, for each mechanism, you need to specify the shared library that implements the services for that mechanism.

The /etc/gss/mech file stores the following information on all installed mechanisms on a system: the mechanism name, in ASCII; the mechanism's OID; the shared library implementing the services provided by this mechanism; and, optionally, the kernel module implementing the service. A sample line might look like this:

kerberos_v5   1.2.840.113554.1.2.2    gl/mech_krb5.so gl_kmech_krb5

For all mechanisms installed, the /etc/gss/qop file stores all the QOPs supported by each mechanism, both as an ASCII string and as its corresponding 32-bit integer.

Both /etc/gss/mech and /etc/gss/qop are created when security mechanisms are first installed on a given system.

Many of the in-kernel RPC routines use non-string values to represent mechanism and QOP. Therefore, applications can use the rpc_gss_mech_to_oid() and rpc_gss_qop_to_num() functions to get the non-string equivalents for these parameters, should they need to maximize use of those in-kernel routines.