Developer's Guide to Oracle Solaris Security

Sending Protected Data in GSS-API

After a context has been established between two peers, a message can be protected before that message is sent.

Establishing a context only uses the most basic GSS-API protection: authentication. Depending on the underlying security mechanisms, GSS-API provides two other levels of protection:

The difference between gss_get_mic() and gss_wrap() is illustrated in the following diagram. With gss_get_mic(), the receiver gets a tag that indicates the message is intact. With gss_wrap(), the receiver gets an encrypted message and a tag.

Figure 4–7 gss_get_mic() versus gss_wrap()

Diagram compares the gss_get_mic and gss_wrap functions.

The function to be used depends on the situation. Because gss_wrap() includes the integrity service, many programs use gss_wrap(). A program can test for the availability of the confidentiality service. The program can then call gss_wrap() with or without confidentiality depending on the availability. An example is Wrapping and Sending a Message. However, because messages that use gss_get_mic() do not need to be unwrapped, fewer CPU cycles are used than withgss_wrap(). Thus a program that does not need confidentiality might protect messages with gss_get_mic().

Tagging Messages With gss_get_mic()

Programs can use gss_get_mic() to add a cryptographic MIC to a message. The recipient can check the MIC for a message by calling gss_verify_mic().

In contrast to gss_wrap(), gss_get_mic() produces separate output for the message and the MIC. This separation means that a sender application must arrange to send both the message and the accompanying MIC. More significantly, the recipient must be able to distinguish between the message and the MIC.

The following approaches ensure the proper processing of message and MIC:

GSS_S_COMPLETE is returned if gss_get_mic() completes successfully. If the specified QOP is not valid, GSS_S_BAD_QOP is returned. For more information, see gss_get_mic(3GSS).

Wrapping Messages With gss_wrap()

Messages can be wrapped by the gss_wrap() function. Like gss_get_mic(), gss_wrap() provides a MIC. gss_wrap() also encrypts a given message if confidentiality is requested and permitted by the underlying mechanism. The message receiver unwraps the message with gss_unwrap().

Unlike gss_get_mic(), gss_wrap() wraps the message and the MIC together in the outgoing message. The function that transmits the bundle need be called only once. On the other end, gss_unwrap() extracts the message. The MIC is not visible to the application.

gss_wrap() returns GSS_S_COMPLETE if the message was successfully wrapped. If the requested QOP is not valid, GSS_S_BAD_QOP is returned. For an example of gss_wrap(), see Wrapping and Sending a Message.

Handling Wrap Size Issues in GSS-API

Wrapping a message with gss_wrap() increases the amount of data to be sent. Because the protected message packet needs to fit through a given transportation protocol, GSS-API provides the function gss_wrap_size_limit(). gss_wrap_size_limit() calculates the maximum size of a message that can be wrapped without becoming too large for the protocol. The application can break up messages that exceed this size before calling gss_wrap(). Always check the wrap-size limit before actually wrapping the message.

The amount of the size increase depends on two factors:

The default QOP can vary from one implementation of GSS-API to another. Thus, a wrapped message can vary in size even if the QOP default is specified. This possibility is illustrated in the following figure.

Diagram shows that the QOP selected affects message size.

Regardless of whether confidentiality is applied, gss_wrap() still increases the size of a message. gss_wrap() embeds a MIC into the transmitted message. However, encrypting the message can further increase the size. The following figure shows this process.

Diagram shows that using confidentiality increases message

GSS_S_COMPLETE is returned if gss_wrap_size_limit() completes successfully. If the specified QOP is not valid, GSS_S_BAD_QOP is returned. Wrapping and Sending a Message includes an example of how gss_wrap_size_limit() can be used to return the maximum original message size.

Successful completion of this call does not necessarily guarantee that gss_wrap() can protect a message of length max-input-size bytes. This ability depends on the availability of system resources at the time that gss_wrap() is called. For more information, see the gss_wrap_size_limit(3GSS) man page.

Detecting Sequence Problems in GSS-API

As a context initiator transmits sequential data packets to the acceptor, some mechanisms allow the context acceptor to check for proper sequencing. These checks include whether the packets arrive in the right order, and with no unwanted duplication of packets. See following figure. An acceptor checks for these two conditions during the verification of a packet and the unwrapping of a packet. See Unwrapping the Message for more information.

Figure 4–8 Message Replay and Message Out-of-Sequence

Diagram shows duplicate and out of sequence error conditions.

With gss_init_sec_context(), an initiator can check the sequence by applying logical OR to the req_flags argument with either GSS_C_REPLAY_FLAG or GSS_C_SEQUENCE_FLAG.

Confirming Message Transmission in GSS-API

After the recipient has unwrapped or verified the transmitted message, a confirmation can be returned to the sender. This means sending back a MIC for that message. Consider the case of a message that was not wrapped by the sender but only tagged with a MIC with gss_get_mic().

    The process, illustrated in Figure 4–9, is as follows:

  1. The initiator tags the message with gss_get_mic().

  2. The initiator sends the message and MIC to the acceptor.

  3. The acceptor verifies the message with gss_verify_mic().

  4. The acceptor sends the MIC back to the initiator.

  5. The initiator verifies the received MIC against the original message with gss_verify_mic().

Figure 4–9 Confirming MIC Data

Diagram shows how message integrity codes are confirmed.

In the case of wrapped data, the gss_unwrap() function never produces a separate MIC, so the recipient must generate it from the received and unwrapped message.

    The process, illustrated in Figure 4–10, is as follows:

  1. The initiator wraps the message with gss_wrap().

  2. The initiator sends the wrapped message.

  3. The acceptor unwraps the message with gss_unwrap().

  4. The acceptor calls gss_get_mic() to produce a MIC for the unwrapped message.

  5. The acceptor sends the derived MIC to the initiator.

  6. The initiator compares the received MIC against the original message with gss_verify_mic().

Applications should deallocate any data space that has been allocated for GSS-API data. The relevant functions are gss_release_buffer(3GSS), gss_release_cred(3GSS), gss_release_name(3GSS), and gss_release_oid_set(3GSS).

Figure 4–10 Confirming Wrapped Data

Diagram shows how wrapped messages with message integrity
codes are confirmed.