Working With Message Segments

This section provides an overview of message segments and discusses how to:

  • Configure nodes to handle segmented messages.

  • Set the maximum number of messages in a message segment.

  • Create message segments.

  • Delete message segments.

  • Send and receive segmented messages between PeopleSoft systems.

  • Send and receive segmented messages to/from third-party systems.

  • Send and receive large segmented messages using parallel processing.

  • Access message segments.

  • View message segment data.

  • Use restartable processing for publishing large messages in batch.

When you create message segments, you can divide rowset-based and nonrowset-based messages into multiple data containers, or segments, for sending. Depending on the order in which you send a message that contains message segments, the receiving system can process the message as a whole, or process one segment at a time while the others are compressed in memory or held in the application database.

As a result creating message segments can enhance system performance and message exchange, especially when you are working with large messages that exceed one gigabyte (1 GB).

To create and manage message segments, you use several methods and properties of the PeopleCode Message class.

This section discusses:

  • Methods used with message segments.

  • Properties used with message segments.

Methods Used with Message Segments

The following table lists the PeopleCode methods you can use when you work with message segments.

Method

Class

Description

CreateNextSegment

Message

Designates the end point of one segment and the beginning of a new segment.

DeleteOrphanedSegments

IntBroker

Used to delete segments that might have been orphaned if you were processing message segments using a PeopleSoft Application Engine program that had to be restarted.

DeleteSegment

Message

Deletes a segment.

GetSegment

Message

Gets the segment specified by the passed value. The passed value is the segment number.

UpdateSegment

Message

Use this method to update data within the current segment.

Note: Use the DeleteSegment and UpdateSegment methods only when storing segments data in memory. These methods do not function when segment data is stored in the database.

Properties Used with Message Segments

The following table lists PeopleCode properties that you can use when you work with message segments.

Property

Class

Description

CurrentSegment

Message

Returns a number, indicating which segment is the current segment.

SegmentsUnOrder

IBInfo

Determines whether to process message segments in order or unordered. This property pertains to asynchronous messages only.

The values are:

  • True: Process message segments unordered.

  • False: Process message segments in order. (Default.)

SegmentCount

Message

Returns the total number of segments in a message.

SegmentsByDatabase

Message

Enables you to override where message segment data is stored for a message.

The values are:

  • True: Store message segments awaiting processing in the application database.

  • False: Store message segments awaiting processing in memory. (Default.)

This section describes how to configure nodes to handle segmented messages.

Understanding Configuring Nodes to Handle Segmented Messages

Before you can send segmented messages, you must configure the remote node defined on the local system to handle segmented messages by setting the Segment Aware option on the Node Definitions page in the PeopleSoft Pure Internet Architecture.

Warning! Do not set the Segment Aware option for remote PeopleSoft 8.45 or earlier nodes, or for third-party systems. If you do so, the receiving system will consume only the first segment of the messages and ignore any subsequent segments.

Configuring a Node to Handle Segmented Messages

To configure a node to handle segmented messages:

  1. Select PeopleTools > Integration Broker > Integration Setup > Node Definitions.

  2. Select a node with which to work and click OK.

    The Node Definitions page appears.

  3. Select the Segment Aware check box.

  4. Click the Save button.

The Maximum Number of Segments parameter is a built-in global parameter that determines the maximum number of segments that can exist in a message. When the number is met, a new message is created and begins to be populated.

The default value is 10.

The Maximum Number of Segments parameter is set on the PeopleTools Options page (PSOPTIONS). To access the page select PeopleTools > Utilities > Administration > PeopleTools Options.

You can also manipulate the maximum number of message segments for a message in PeopleCode using the %MAXNBRSEGMENTS built-in function.

This section provides an overview of creating message segments and message segment numbers and discusses how to:

  • Create message segments.

  • Count the number of segments in messages.

  • Store message segments awaiting processing.

  • Override where to store message segment awaiting processing.

  • Specify the order in which to process message segments.

  • Chunk asynchronous segmented messages.

Understanding Creating Message Segments

By default every message has one segment.

To create multiple message segments use the CreateNextSegment method in the location in the message where you want one segment to end and next segment to begin. Continue this process until you have created the desired number of segments for the message.

Segments can contain any number of rowsets of data (rowset-based messages) or rows of data (nonrowset-based messages).

Understanding Message Segment Numbers

When you create a message segment, PeopleSoft Integration Broker assigns a message segment number to the segment.

The first message segment has a message segment number or 1, and message segment numbers are increment by one sequentially thereafter. As an example, if you break a message into three segments, the first segment number is 1, the second segment number is 2, and the third segment number is 3.

Creating Message Segments

The following example shows using the CreateNextSegment method to create three segments in the message QE_FLIGHTPLAN, populating each segment with data from the component buffer.

&MSG = CreateMessage(OPERATION.QE_FLIGHTPLAN);

&rs=&MSG.GetRowset();
//Now populate rowset
// End of first segment. Beginning of second segment.
&MSG.CreateNextSegment(); 

&rs=&MSG.GetRowset();
//Now populate rowset
//End of second segment. Beginning of third segment.
&MSG.CreateNextSegment(); 

&rs=&MSG.GetRowset();
//Now populate rowset

%IntBroker.Publish(&MSG);

Counting the Number of Segments in Messages

You might have the need to determine the number of segments in a message. Use the SegmentCount property to determine this information.

Storing Message Segments Awaiting Processing

By default, message segments awaiting processing are stored in memory until all segments are processed. Once all segments are processed, PeopleSoft Integration Broker sends all data as one message.

Use the MessageSegmentFromDB parameter in PSAdmin to specify the number of segments to keep in memory before writing segmented messages to the database. The default value is 10.

For synchronous messages, if the number of segments sent for processing exceeds the set for the MessageSegmentsFromDB parameter, an error occurs.

Overriding Where to Store Message Segments Awaiting Processing

You can override the number of segments to keep in memory before writing segmented messages to the database for a single message using the SegmentsByDatabase property of the Message class.

Storage Location

Description

Memory

When message segments are stored in memory, PeopleSoft Integration Broker writes all segments as one message to the database when you send the message.

To store message segment data in memory, set the SegmentsByDatabase property to False. (Default.)

Application database

When message segments are stored in the database, PeopleSoft Integration Broker writes the segments to the database individually. When you store message segments in the database you can have an infinite number of segments in a message.

To store message segment data in the application database, set the SegmentsByDatabase property to True.

When you store message segments in memory, the number of segments is limited by the value set in the MessageSegmentFromDB parameter in PSAdmin in the Setting for PUB/SUB servers section of the file.

When working with asynchronous messages, if you create more message segments then the value set, all segments are written to the database automatically and the SegmentsByDatabase property will automatically be set to True.

For synchronous messages, attempting to create more segments then the specified value will result in an error message.

Specifying the Order in Which to Process Message Segments

When you work with segmented asynchronous messages you can specify that PeopleSoft Integration Broker process the segments in order or unordered, using the SegmentsUnOrder property of the Message class.

Message Segment Processing

Description

In order

When Integration Broker processes message segments in order, it decompresses all message segments sequentially and then processes the message as a whole. In this situation, only one publication or subscription contract is created.

To process message segment in order, set the SegmentsUnOrder property to False.

Unordered

When Integration Broker processes message segments unordered, it decompresses and processes all segments in parallel. In this situation, the system creates one publication or subscription contract for each message segment.

To process message segment unordered, set the SegmentsUnOrder property to True.

If you attempt to send ordered segmented messages to a node that is not segment aware an error message will be created and can be viewed on the Message Errors tab on the Message Details page in Service Operations Monitor.

See the product documentation for Integration Broker Service Operations Monitor.

Chunking Asynchronous Segmented Messages

Chunking asynchronous segmented messages sends message in blocks to the receiving node.

When using chunking, message instances display in Hold status in the Service Operations Monitor until all chunks are received. Once all chunks are received, the message status switches to New.

Note: Chunking applies to ordered asynchronous messages only.

The number of segments to chunk for an asynchronous message is determined by the value you set for the MessageSegmentByDatabase parameter in PSAdmin. The default value is 10.

As an example, if a message has 20 segments and you set MessageSegmentByDatabase to 5, PeopleSoft Integration Broker will send four groups (array of messages) of segments to the integration gateway, and each group will contain five segments.

You can delete message segments in a message only before you publish the message.

Use the DeleteSegment method of the Message class to perform the action.

You cannot delete the first segment in a message.

The following example demonstrates using the DeleteSegment method in an implementation of the OnRequestSend method.

import PS_PT:Integration:ISend;

class Send implements PS_PT:Integration:ISend
   method Send();
   method OnRequestSend(&message As Message) Returns Message;
   method OnError(&message As Message)
end-class;

/* constructor */
method Send
   %Super = create PS_PT:Integration:ISend();
end-method;

method OnRequestSend
   /+ &message as Message +/
   /+ Returns Message +/
   /+ Extends/implements PS_PT:Integration:ISend.OnRequestSend +/
   Local integer &segment_number, &i;
   Local Rowset &rs;
   
   For &i = 1 To &message.SegmentCount
      &rs = Null;
      &message.GetSegment(&i);
      
      &rs = &message.GetRowset();
      
      /* determine that segment 3 needs to be deleted. */
      &segment_number = &i;
      
   End-For;
   
   &message.DeleteSegment(&segment_number);
   
   
   Return &message;
end-method;

method OnError
   /+ &message as Message +/
   /+ Extends/implements PS_PT:Integration:ISend.OnError +/
   
end-method;

This section discusses how to:

  • Send segmented messages to PeopleSoft systems.

  • Receive segmented messages from PeopleSoft systems.

Sending Segmented Messages to PeopleSoft Systems

To send a segmented message, use sending PeopleCode and events as you would with any other message.

Use the PeopleSoft target connector when the receiving node is a PeopleSoft system. The PeopleSoft target connector automatically handles message segments, and no additional configuration is required on the connector.

Before sending a transaction with message segments, on the sending PeopleSoft system, be sure that the Segment Aware check box is selected for the remote node that represents the receiving system.

Receiving Segmented Messages from PeopleSoft Systems

To receive segmented message from PeopleSoft systems, use notification PeopleCode or implement the OnRequest method.

Use the PeopleSoft listening connector to receive transactions that contain message segments from other PeopleSoft systems. The PeopleSoft listening connector automatically handles message segments, and no additional configuration is required on the connector.

This section discusses how to:

  • Send segmented messages to third-party systems.

  • Receive segmented messages from third-party systems.

Understanding DataChunkCount and DataChunk Properties

PeopleSoft Integration Broker uses two properties to communicate to sending and receiving systems the number of message segments that are contained in a transaction:

Field or Control

Definition
DataChunkCount

Indicates the total number of data chunks or message segments contained in the transaction.
DataChunk

Indicates the number of the data chunk or message segment that you are sending.

For example, if there are a total of seven data chunks in the transaction, and the current segment is the third chunk, the DataChunk value for the current message is 3.

Note that when you are sending and receiving message segments between PeopleSoft systems these properties are not used. The PeopleSoft target and listening connectors perform all necessary processing.

Sending Segmented Messages to Third-Party Systems

To send segmented messages from PeopleSoft systems to third-party system, use one of the following target connectors:

  • AS2 target connector

  • HTTP target connector

  • JMS target connector

  • SMTP target connector

No additional target connector configuration is required to send segmented messages. These connectors read the messaging PeopleCode on the integration gateway and determine the number of segments contained in the transaction. They then populate the DataChunkCount and DataChunk parameters and include this information with each outbound segment sent. All of these connectors except for the HTTP target connector send the DataChunkCount and DataChunk information in the message header of each outbound message segment. The HTTP target connector includes the DataChunkCount and DataChunk parameter information in the HTTP header of each outbound message segment.

Before sending a transaction with message segments, on the PeopleSoft system, be sure that the Segment Aware check box is selected for the remote node that represents the third-party integration partner.

Receiving Segmented Messages from Third-Party Systems

At this time, only the HTTP listening connector can be used to receive message segments from third-party systems.

To receive segmented messages with third-party integration partners, the third-party must specify the following DataChunkCount and DataChunk parameters in the HTTP properties, query arguments, or SOAP header:

The receiving PeopleSoft system must use the HTTP listening connector as only this connector monitors transactions for these parameters.

After the third party sends in the first segment, the PeopleSoft system sends an acknowledgement to the third-party system. The acknowledgment contains a transaction ID that the third-party integration partner must include with all subsequent segments.

The following bullet points describe sample processing for a third-party integration partner sending a transaction to a PeopleSoft system that contains three segments:

  1. First segment processing:

    1. The third-party integration partner prepares the first message/segment of the transaction. In the HTTP properties, query string, or SOAP header, it sets the DataChunk equal to 1 indicating the first chunk, and sets the DataChunkCount equal to 3 indicating total number of chunks to be sent for the transaction.

    2. When the request is received by the PeopleSoft system the data chunk is saved in the database as a segment.

    3. In the Service Operations Monitor the transaction displays a status of Hold.

    4. The PeopleSoft system sends an acknowledgement to the third-party system, which includes a transaction ID.

      Note: The third-party integration partner must include the transaction ID as part of all subsequent requests for the transaction. The PeopleSoft system uses the transaction ID to identify the segments that belong to the transaction.

  2. Second segment processing:

    1. The third-party integration partner prepares the second message/segment of the transaction. In the HTTP properties, query string, or SOAP header, it sets the DataChunk equal to 2 indicating that the message is the second chunk, and sets the DataChunkCount equal to 3 indicating total number of chunks to be sent for the transaction. It also specifies the transaction ID sent by the PeopleSoft system in the acknowledgement for the first segment.

    2. When the request is received by the PeopleSoft system the data chunk is saved in the database as a segment.

    3. In the Service Operations Monitor the transaction displays a status of Hold.

  3. Third segment processing:

    1. The third-party integration partner prepares the third message/segment of the transaction. In the HTTP properties, query string, or SOAP header, it sets the DataChunk equal to 3 indicating that the message is the third chunk, and sets the DataChunkCount equal to 3 indicating total number of chunks to be sent for the transaction. It also specifies the transaction ID sent by the PeopleSoft system in the acknowledgement for the first segment.

    2. When the request is received by the PeopleSoft system the data chunk is saved in the database as a segment.

    3. Since the PeopleSoft system has received all of the segments in the transaction, in the Service Operations Monitor the transaction displays a status of New.

    4. The PeopleSoft system processing the transaction like any other transaction at this point.

The PeopleCode to read the data chunks/segments is the Message Segment API.

Previous sections in this topic have discussed sending one message that contains multiple message segments. For very large messages this can have performance impact due to the large number of segments.

PeopleSoft provides PeopleCode that allows you to send multiple messages with multiple segments and then correlate them into one transaction on the receiving system. So instead of sending one message with 50 message segments, you can send 10 messages with 5 message segments using parallel processing, and then correlate the 10 messages on the receiving system.

The InitializeConversationId property on the Message object, provides the correlation between messages. The FirstCorrelation method on the Message object ensures that the database table is truncated after receipt of the first message only.

On the first message to be published set the InitializeConversationId property to True. After the message is published retrieve the transaction ID from the message. For all subsequent messages, set the CorrelationID property to the value of the transaction ID returned from the first message. As a result, when the messages arrive at the receiving system they have different transaction IDs, but all have the same correlation ID.

On the receiving system when the first message is received the database table is truncated. To prevent a destructive load from occurring with the receipt and processing of each subsequent message, use the PreNotify event. You can use the PreNotify event to truncate the database table upon receipt of the first message. In subsequent messages use the FirstCorrelation method in the event, setting the method to True, to determine if a prior message with the same correlation ID has already run the event.

The following example shows an example of how the sending system uses the InitializeConversationId property:

/*First Message to Publish */
	
&MSG. InitializeConversationId = true;
%IntBroker.Publish(&MSG);
&strCorrelationID  =  &MSG.TransactionId;

/* all subsequent message to correlate*/

&MSG. IBInfo.ConversationID = &strCorrelationID;
%IntBroker.Publish(&MSG);

The following example shows an example of how the receiving system uses the FirstCorrelation method in the PreNotify event:

PreNotify Event:

If &MSG.FirstCorrelation() = true Then
/* process the event logic */

End-If;

Accessing message segments is described elsewhere in this section.

See Accessing Segments in Messages.

After you receive a segmented message, use the GetSegment method of the Message class to access message segment data.

After you access a message segment, use the Message class GetRowset or GetXmlDoc methods to work with the contents of the segment.

Warning! You can access only one segment in a message at a time. When you access a message segment, PeopleSoft Integration Broker removes the previously accessed message segment from memory.

When you access a message segment, set the existing rowset to null to eliminate storing multiple rowsets in the data cache.

The following example shows using the GetSegment method to access a message segment in the message QE_FLIGHTDATA.

For &i = 1 To &MSG.SegmentCount
   &rs = Null;  //Null the rowset to remove it from memory
   &MSG.GetSegment(&i);
   
   &rs = &MSG.GetRowset();
   &REC = &rs(1).QE_FLIGHTDATA;
   
   &FLIGHTDATA = CreateRecord(Record.QE_FLIGHTDATA);
   &REC.CopyFieldsTo(&FLIGHTDATA);
   
   /* Parse out Message Data */
   &acnumber_value = &FLIGHTDATA.QE_ACNUMBER.Value;
   &msi_sensor_value = &FLIGHTDATA.QE_MSI_SENSOR.Value;
   &ofp_value = &FLIGHTDATA.QE_OFP.Value;
   &actype_value = &FLIGHTDATA.QE_ACTYPE.Value;
   &callsign_value = &FLIGHTDATA.QE_CALLSIGN.Value;
   &squadron_value = &FLIGHTDATA.QE_SQUADRON.Value;
   &comm1_value = &FLIGHTDATA.QE_COMM1.Value;
   &comm2_value = &FLIGHTDATA.QE_COMM2.Value;
   &ecm_value = &FLIGHTDATA.QE_ECM.Value;
   
   &outstring = "Send Async Flight test";
   
   /* Construct Output String */
   &outstring = &outstring | &acnumber_value | &CRLF | 
   &msi_sensor_value | &CRLF | &ofp_value | &CRLF | &actype_value | 
   &CRLF | &callsign_value | &CRLF | &squadron_value | &CRLF | 
   &comm1_value | &CRLF | &comm2_value | &CRLF | &ecm_value;
   
   /* Log Output String into page record */
   &FLIGHTDATA.GetField(Field.DESCRLONG).Value = &outstring;
   
   SQLExec("DELETE FROM PS_QE_FLIGHTDATA");
   &FLIGHTDATA.Insert();
   
End-For;

The Service Operations Monitor Message Details page provides information about messages that contain segments.

This section provides an overview, prerequisites and setup steps for using restartable processing for publishing large asynchronous segmented messages in batch.

Understanding Using Restartable Processing

PeopleSoft provides a PeopleSoft Application Engine library module, IB_SEGTEST, that you can use as a template to create a module to aid in processing large messages and messages in batch for outbound asynchronous PeopleSoft Integration Broker segment data with restart capability.

With restart capability, if there is an abnormal program termination, you can correct any data errors and continue processing from the point of the last commit without having to reload message segment data from the beginning.

Understanding the IB_SEGTEST Application Engine Library Module

This section provides overview information for using the IB_SEGTEST

The IB_SEGTEST library module consists of three sections:

  • Section 1: Section1. The main processing section.

  • Section 2: ABORT. Use to trigger a user abort of the running application engine program

  • Section 3: CLEANSEG. An independent section you can call to clean up pending segment data that had been committed to the database but is no longer to be used.

Prerequisites

To use the information provided in this section, you should have a thorough understanding of PeopleSoft Application Engine.

Using the IB_SEGTEST Library Module

This section provides an overview of the high-level list of tasks to perform to set up a PeopleSoft Application Engine program to perform restartable message processing.

  1. Make a copy of IB_SEGTEST, including all sections and PeopleCode.

    From here on, the copy of the application engine library module is referred to as IB_SEGTEST1, but you can use any name you choose.

  2. In the State Records tab of IB_SEGTEST1, verify that PSIBSEGRSTR_AET is the default state record. Replace PT_EIP_ERR_AET with whatever state record is used in the main application engine program that will be calling the Library module.

    Note that IB_SEGTEST1 is flagged as not restartable. Since database commits will be performed in the middle of PeopleCode processing, the only way the commits can take effect is if the module is flagged as not restartable.

  3. The application engine program used to call IB_SEGTEST1 should be restartable.

    Always issue a commit in the step prior to calling the library module IB_SEGTEST1.

  4. In the application engine program that will be calling IB_SEGTEST1, insert a step to call IB_SEGTEST1, section Section1. Insert the step at the point in time when you want to do the message publish. You must issue a commit prior to calling this section, otherwise there will be a ‘Unable to Process Commit’ error issued from within IB_SEGTEST1.

  5. Add PSIBSEGRSTR_AET as an additional state record to the calling application engine program.

  6. Since both programs now share state records, when IB_SEGTEST1 is called, all state record values will be passed on to the called module. Presumably all application values needed to extract application data would be stored in the application state record.

  7. Modify the PeopleCode in IB_SEGTEST1.Section1. Several comments have been added to the code to aid in the modifications. Note the following:

    • Change &MSG = CreateMessage(OPERATION.QE_FLIGHTPLAN) to create whatever message will be used.

    • SegmentsByDatabase should always be set to True.

    • The While loop is used to simulate application code processing large volumes of data. This can be changed to meet application needs. However, pay close attention as to when commits are issued, when state records are updated, when new segments are created, and finally, when the message publish is executed. The order of these events is crucial to proper workability. In the sample program, also note how to break out of the While loop.

    • Note the location where the application state record needs to be updated. A comment instructs in the PeopleCode provides instructions on where to perform this task.

    • Do not remove the Exit(1) from the end of the PeopleCode. This is necessary to bypass the Abort action that is coded into the same Step.

    • If in the middle of processing, the application code determines that an abort needs to be triggered, an Exit(0) can be coded. This triggers the Abort step to be called, which will terminate application engine processing. A restart could then be issued if processing needs to continue.

      If you determine that a message no longer needs to be published, the calling application engine program could then call the CLEANSEG step to get rid of all the pending data that has been saved in the database. Alternatively, the Abort step could be modified to call CLEANSEG if on any abort, no old data is to be kept.

See the product documentation for Application Engine for more information about application engine programs.