3 Upgrade Considerations

This chapter provides considerations for upgrading various Oracle Service Bus configuration artifacts to Oracle Service Bus 11g Release 1 (11.1.1.7.0).

It includes the following topics:

3.1 Upgrade Considerations for AquaLogic Service Bus 2.6 Users

Read the following sections if you are using AquaLogic Service Bus 2.6 configurations:

3.1.1 Integrated Development Environment

Many of the design time features available in the AquaLogic Service Bus Console are available in Oracle Enterprise Pack for Eclipse, which is the Oracle Service Bus integrated development environment (IDE).

If you want to use the IDE instead of the Console, you can import an AquaLogic Service Bus 2.6 configuration JAR directly into the 11g Release 1 (11.1.1.7.0) IDE. For information about importing a JAR file into the 11g Release 1 (11.1.1.7.0) IDE, see the "Importing Resources" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

Note:

When you import any configuration JAR into the 11g Release 1 (11.1.1.7.0) IDE, the operational and administrative settings are removed. To retain these settings, first import the configuration JAR file into the console, export and import the file into the 11g Release 1 (11.1.1.7.0) IDE, as described above. When you later move configuration from IDE to Console, enable Preserve operational settings, so that operational settings that were imported in the first step are preserved.

For information about exporting JAR files, see Task 2: Exporting Security Configurations.

3.1.2 Alert Rules

The Service Level Agreements (SLA) alert rules features in AquaLogic Service Bus 3.0 and later differ slightly from previous releases. These changes do not affect the run-time evaluation or how alerts are issued. However, you may notice the following changes:

  • In AquaLogic Service Bus 2.6, alert rule resources were created as separate resources and individually maintained. In Oracle Service Bus 11g Release 1 (11.1.1.6.0), alert rules are part of the service definition. Because Alert Rules are part of the service definition and are no longer resources themselves, this affects their display in the References and Referenced By pages in the AquaLogic Service Bus Console. For information about viewing references, see the "View References Page" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

  • You may notice minor changes in the content of the alerts as related to various destinations. For information about alert destinations, see the "Alert Destinations" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

  • Starting in AquaLogic Service Bus 3.0, if an alert rule is renamed, then for the alerts issued in the past and under the old name, the console will no longer provide access to the rule definition on the Alert Summary and Extended SLA Alert History pages.

3.1.3 Displaying References from Alerts to Alert Destinations

In AquaLogic Service Bus 2.6 and earlier versions where you could see distinct entries of alert rules in the AquaLogic Service Bus Console, references to alert destinations through alert rules from proxy and business services are maintained and displayed as a single reference. For example, in a proxy service, if multiple alert rules and multiple pipeline alert actions use the same alert destination, only one entry for the alert destination is displayed in the Referenced By page for that alert destination. For more information, see the "Alert Destinations" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

Since Alerts are no longer separate resources in AquaLogic Service Bus 3.0 and later releases, the way references from alerts to alert destinations are displayed is different from AquaLogic Service Bus 3.0 and later releases. Two pages in the Console are affected. First, the Referenced By field in the Alert Destination page no longer displays the Alert Rule that is referencing the destination. Instead, the Reference By field displays the service that contains the alert. A side effect of this is that if a service has multiple alerts (SLA alerts or pipeline alerts) that reference the same Alert destination, the associated service is listed only once in the Referenced By field. Second, the Alert Rule page no longer contains the Reference information. Instead, the Service Summary page for the associated service contains the Alert Destinations referenced in the References field. For more information, see the "Alert Destinations" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

3.1.4 Details Sent to Alert Destinations

When an SLA alert is issued to configured destinations, the alert details include a Rule ID as in AquaLogic Service Bus 2.6 and before. However, in AquaLogic Service Bus 3.0 and later releases, the value of the Rule ID is set to a combination of the global name of the parent service and the name of the alert rule. For the SNMP trap, the Rule ID is truncated to 64 characters, as in AquaLogic Service Bus 2.6 and earlier.

3.1.5 Import-Export Alert Rule Changes

Unlike AquaLogic Service Bus 2.6, which merged alerts with any existing alerts during import, AquaLogic Service Bus 3.0 and later releases provides users with preserve-overwrite semantics. This feature allows you to either keep all existing alerts or overwrite them, regardless of whether the alerts have the same name or not.

3.1.6 Session-Aware Access Control Management of Proxy Services

The security realm must be configured before completing the steps described in this section.

The exported JARs from AquaLogic Service Bus 2.6 do not contain any access control policies. Before importing a configuration JAR from AquaLogic Service Bus 2.6 release, Oracle Service Bus 11g Release 1 (11.1.1.7.0) uses a pre-import access control policy to perform an in-place upgrade of the JAR to 11g Release 1 (11.1.1.7.0). To perform the in-place upgrade, the pre-processor queries all manageable authorization providers for each proxy service and retrieves a list of applicable access control policies. It then inserts those policies into the service definition of the proxy service. This is done on best-effort basis.

A manageable authorization provider is an authorization provider that implements the PolicyEditorMBean interface. Such providers expose read-write APIs that allow the Oracle WebLogic Server and the Oracle AquaLogic Service Bus console to add, modify, or delete policies stored in them.

For transport level and default message-level policies, the system queries only those providers that expose the PolicyEditorMBean to retrieve any applicable policies, and inserts these policies into the service definition.

For operational message-level policies, the system can query providers that have implemented the PolicyListerMBean. For providers that have not implemented the PolicyListerMBean interface, the operation-level policies are not retrieved.

After the in-place upgrade finishes, the import process proceeds as if the configuration JAR is of type 11g Release 1 (11.1.1.7.0), including the access control policies retrieved from the authorization providers. Table 3-1 explains various combinations of the applicable parameters and the outcome of the import process. Note that the outcome is the result of the import process and does not represent anything done after the configuration is imported.

Table 3-1 Applicable Parameters and Import Outcomes

Version of config.jar Being Imported Proxy ACLs in Core Repository Preserve Policies ACLs in Session Service Definition Explanation

2.6

New

N/A (No)

N/A (No)

From manageable authorization providers

Upgrades the JAR to 11g Release 1 (11.1.1.7.0), which involves pulling all applicable ACLs from all manageable authorization providers. The service definition in session will have ACLs from the configuration JAR directly from the Authorization Providers.

2.6

Exists

No

No

From manageable authorization providers

Upgrades the JAR to 11g Release 1 (11.1.1.7.0). The service definition in session has ACLs from the configuration JAR directly from the Authorization Providers.

2.6

Exists

No

No

None

Upgrades the JAR to 11g Release 1 (11.1.1.7.0). The service definition in session has no ACLs.

2.6

Exists

Yes

No

From manageable authorization providers

Upgrades the JAR to 11g Release 1 (11.1.1.7.0). The service definition in session has ACLs from the configuration JAR directly from the Authorization Providers.

2.6

Exits

Yes

Yes

From the core repository in the config framework

Upgrades the JAR to 11g Release 1 (11.1.1.7.0). The service definition in session retains ACLs from the core repository


If an authorization provider does not exist in the target Oracle Service Bus 11g Release 1 (11.1.1.7.0) system, the import process ignores the imported ACLs for the authorization provider and displays a warning. In this case, you can discard the session, or undo the import task, and then add the authorization providers to the server and re-import. Alternatively, you can do a dummy update operation of security parameters in the Oracle Service Bus Console, and the system will auto-correct any conflicts that it can on best-effort basis. These changes are atomic and reversible if you discard the session.

For more information about updating the security parameters, see the "Message Level Security Configuration" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

3.1.7 Transport SDK and Transport Provider Changes

The following changes may impact the Oracle Service Bus configuration:

3.1.7.1 Message Retry Count for Business Service Configuration

In AquaLogic Service Bus 2.6, the message retry count applies to the list of URIs for a business service. In Oracle Service Bus 11g Release 1 (11.1.1.7.0), the retry count applies to the individual URL endpoints. The upgrade process maintains the 2.6 behavior as follows:

new_retries = N-1 + old_retries*N

where N is the total number of URIs and old_retries is the 2.6 retry count.

For example, suppose that in AquaLogic Service Bus 2.6, you have three URLs configured for the business service and a retry count of one. With the 2.6 retry mechanism all three URLs are tried. Then after the retry delay, all three URLs are retried again. To obtain the same behavior in 3.0 and later releases, the retry count is changed to five, which is obtained by applying the formula: (3 -1) + (1*3) = 5. The net effect is exactly the same: all three URLs are tried once (using two of the five retries), then after the retry delay, the three URLs are tried once more (using the last three of the five retries).

If only a single URL is configured, the old behavior and the new behavior are the same; the retry count does not change during the upgrade.

For more information, see the "Business Services: Creating and Managing" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

3.1.7.2 Duplicate URIs for a Business Service are Removed

When importing business services into Oracle Service Bus 11g Release 1 (11.1.1.7.0), the import process removes the duplicate URIs in the 2.6 configurations. If the URIs use randomly weighted load balancing algorithms and the weights are set, the weights are adjusted accordingly. For example, if the business service is configured with the following URIs and weights:

  • URI_A 1

  • URI_B 3

  • URI_A 1

When the business service is upgraded into 11g Release 1 (11.1.1.7.0), the URI set is modified as follows:

  • URI_A 2

  • URI_B 3

For Business services configured with other algorithms, the upgrade removes the duplicate URIs and no other changes are made.

For more information about setting the parameters for the load balancing algorithm, see the "Business Services: Creating and Managing" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

3.1.7.3 Application Errors Retries

In case of delivery failure when sending outbound requests, Oracle Service Bus allows you to specify whether to retry endpoint URIs for application errors, such as a SOAP fault. This does not affect retries for communication errors. This new option is available on the Transport Configuration page for business services. For more information, see the "Transport Configuration Page" section in the Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus. After the number of retries is exceeded, an error is issued.

To maintain the 2.6 behavior, new tag is added with the default value set to true.

<xs:element name="retry-application-errors" type="xs:boolean"  minOccurs="0"/>

This tag will only be added to the transport end point whose provider configuration has the flag declare-application-errors as true.

3.1.7.4 HTTPS Transport Changes

To simplify switching between HTTP and HTTPS in the AquaLogic Service Bus 3.0 and later releases console, the HTTPS transport configuration has been removed and its functionality has been added to the 3.0 inbound HTTP transport provider. The 11g Release 1 (11.1.1.7.0) HTTP Transport Configuration page contains a check box to enable HTTPS.

A new element, use-https is added to the schema of the HTTP Transport inbound properties.

<xs:element name="use-https" type="xs:boolean" minOccurs="0"/>

Any existing HTTPS Transport configurations are upgraded to HTTP transport with this flag set to true.

Note:

This functionality is only applicable to HTTP proxy services.

3.1.7.5 Transport Configuration in the Design Environment

In earlier releases, AquaLogic Service Bus transports were configured only through the AquaLogic Service Bus Console. However, starting in ALSB 3.0, the transports can be designed on Eclipse. For more information, see "Developing Oracle Service Bus Transports for Workshop for WebLogic" in the Oracle Fusion Middleware Transport SDK User Guide for Oracle Service Bus.

3.2 AquaLogic Service Bus 3.0 Upgrade Considerations

Read the following sections if you are using AquaLogic Service Bus 3.0:

3.2.1 Upgrading an AquaLogic Service Bus 3.0 Workspace in Oracle Enterprise Pack for Eclipse

To upgrade an AquaLogic Service Bus 3.0 workspace to Oracle Service Bus Oracle Enterprise Pack for Eclipse, perform the following steps:

Note:

After you upgrade the workspace, you can no longer open it in WorkSpace Studio 1.1 for ALSB 3.0.

  1. Start WorkSpace Studio 1.1 in ALSB 3.0 and open the workspace you are upgrading.

  2. Close the ALSB perspective and all editors.

  3. Close all projects.

  4. Close WorkSpace Studio 1.1.

  5. Back up your workspace.

  6. Start Oracle Enterprise Pack for Eclipse in 11g Release 1 (11.1.1.6.0), and open the workspace you are upgrading.

  7. Wait for upgrade to start. When you open an AquaLogic Service Bus 3.0 workspace in Oracle Enterprise Pack for Eclipse for Oracle Service Bus 11g Release 1 (11.1.1.7.0), it will take a few moments for the upgrade process to start. Do not edit the workspace until the upgrade completes and the confirmation dialog appears.

  8. After upgrading projects, open the Oracle Service Bus perspective and continue working with the newly upgraded projects and artifacts.

3.2.2 JNDI Service Account Deprecated for Java Message Service Business Service

In Oracle Service Bus 11g Release 1 (11.1.1.7.0), the jndi-service-account is deprecated. After upgrading to 11g Release 1 (11.1.1.7.0), the Oracle Service Bus Java Message Service (JMS) business service uses the jms-service-account for both JMS and JNDI purposes.

The following table shows how the Oracle Service Bus JMS business service migrates JNDI and JMS accounts.

Table 3-2 JMS Business Service JMS and JNDI Account Migration

jms-service-account Before Upgrade jndi-service-account Before Upgrade jms-service-account After Upgrade

sa-1

sa-1

sa-1

sa-1

sa-2

sa-1

sa-1

sa-1

sa-2

sa-2


3.2.3 Pipeline Action ID Upgrade

After upgrading to 11g Release 1 (11.1.1.7.0), all the actions in a pipeline are assigned a unique ID.

3.2.4 Pipeline Monitoring Level

In 11g Release 1 (11.1.1.7.0), proxy services use a new monitoring flag to control the level of statistics collected. The three levels are:

  • Service - coarse grained statistics.

  • Pipeline - the same level at which statistics were gathered in AquaLogic Service Bus 3.0.

  • Action - fine grained statistics.

The upgrade sets the monitoring flag to a value of Pipeline.

3.2.5 Enhanced Validation

As of Oracle Service Bus 11g Release 1 (11.1.1.7.0), validation rules in the following areas have been made more strict and may result in design time or run time errors:

  • Enhanced WSDL and service validation: after upgrading from previous releases to Oracle Service Bus 11g Release 1 (11.1.1.7.0), one may get conflict messages, such as:

    • [OSB Kernel:398022]No corresponding mapping was found in the service definition for the WSDL operation: OPERATION-NAME

    • [OSB Kernel:398034]Two operations expect the same incoming message, you must use a selector different than message body

    In this case, update your WSDL or service as the error message indicates.

  • Enhanced Split-Join validation: AquaLogic Service Bus 3.0 allowed an insert action in a Split-Join to insert into an uninitialized variable. In Oracle Service Bus 11g Release 1 (11.1.1.7.0), such an insert action will fail with the following error:

    Variable 'VARIABLE-NAME' is being referenced before it has been initialized with a value.: Fault [{http://docs.oasis-open.org/wsbpel/2.0/process/executable}uninitializedVariable]
    

    If your Split-Join works in AquaLogic Service Bus 3.0 and fails in Oracle Service Bus 11g Release 1 (11.1.1.7.0) with the above error, then modify the Split-Join to initialize the variable before insert.

  • Enhanced JMS proxy and business service URI validation: prior to Oracle Service Bus 11g Release 1 (11.1.1.7.0), you could enter a JMS proxy and business service URI without a host and port. In Oracle Service Bus 11g Release 1 (11.1.1.7.0), Oracle Service Bus applies the following validation:

    • For a JMS proxy service URI, you may omit the host and port. Doing so will display a confirmation dialog prior to commit.

    • For a JMS business service URI, you must always specify the host and port.

3.3 Oracle Service Bus 10g Upgrade Considerations

You must read the following upgrade considerations if you are using Oracle Service Bus 10g Release 3 Maintenance Pack 1 (10.3.1) or Oracle Service Bus 10g Release 3 (10.3):

3.3.1 Alerts Upgrade

The upgrade will preserve the existing behavior as follows:

  • An Alert Log Enabled flag is added to the Alert destinations and the default value is true, by default.

  • SLA definitions and Pipeline Alert definitions that do not have an alert destination associated with them will be upgraded as follows:

    • A new alert destination is created as part of the upgrade process. It is created under the same project as the service being upgraded, and it will have the name AlertDestinationForLogging. This alert destination will have only alert logging turned on. If there are multiple services under the same project that are subject to this upgrade logic, they will share the same alert destination.

    • SLA and pipeline alert definitions with no alert destinations are upgraded, so that they now reference this new alert destination.

3.3.2 JCA Endpoint Configuration

JCA Endpoint configuration will be upgraded to use the new configuration. WSDL with JCA WSIF extensions will be upgraded to Oracle Service Bus 11g adapter artifact. For example, JCA file, abstract WSDL, and concrete WSDL will provide the interface to upgrade JCA 10g wsdl to WSDL 11g and JCA file. Other upgrades will be as follows:

Config JAR Upgrade

JCA upgrader will scan for the imported Oracle Service Bus config.jar file, and then the JCA WSDL (Oracle SOA 10g WSDL with WSIF JCA extension) and the JCA services in the config.jar file will be upgraded.

Note:

Only JCA WSDL that are associated with JCA services will be upgraded. If JCA WSDL is not used by any JCA service, then it will not be upgraded.

JCA WSDL Upgrade

JCA WSDL in the imported config.jar file will be upgraded. A JCA resource and an abstract WSDL will be generated from the JCA WSDL. A concrete WSDL will be generated from the abstract WSDL and the JCA resource. The concrete WSDL will be generated based on the following rules:

  • Target namespace should be the same as the abstract WSDL.

  • JCA (concrete) WSDL in 10g with the WSIF JCA binding will be upgraded to a JCA resource containing the JCA binding, abstract WSDL containing the abstract part of the 10g WSDL, and a concrete WSDL containing the soap binding based on portType defined in the abstract WSDL. The JCA service based on the JCA WSDL in 10g will be upgraded to be based on the concrete WSDL containing the soap binding and will also have a reference to the JCA resource.

  • The binding section will contain a SOAP 1.1 binding.

  • The SOAP 1.1 binding style is document.

  • The SOAP 1.1 binding section will contain all operations from the abstract WSDL.

  • For each binding operation, a SOAP 1.1 operation element will be generated with soapAction attribute set to the operation name.

  • For each binding operation's input and output element, there will be a SOAP 1.1 body element generated with attribute use set to literal.

  • To support specific AQ use case that has header message defined in abstract WSDL, a SOAP header element will be generated for the operation's input binding section. The assumption is that AQ abstract WSDL will contain a WSDL message name “Header_msg” and the message contains a single part named “Header”.

  • A service section will be created with a port for the generated binding The generated port will contain a SOAP 1.1 address element with location attribute set to:

    jca://<adapter_connection_factory_jndi>
    

Note:

If a non-jca (e.g. http, jms etc.) service type proxy/business service in Oracle Service Bus 10g is based on a WSDL that has jca binding, and if the same WSDL is not used by a jca service type proxy/business service, then the WSDL will not be upgraded and hence there will be a conflict on import. You must manually fix the WSDL to have a non-jca binding such as soap binding to over come the conflict.

Example 3-1 JCA Concrete WSDL

<?xml version="1.0"?>
<definitions name="db-inbound-concrete"
      targetNamespace="http://xmlns.oracle.com/pcbpel/adapter/db/db_inbound/"
      xmlns:tns="http://xmlns.oracle.com/pcbpel/adapter/db/db_inbound/"
      xmlns="http://schemas.xmlsoap.org/wsdl/"
      xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
  <import namespace="http://xmlns.abc.com/pcbpel/adapter/db/db_inbound/" 
          location="inbound.wsdl"/>      
  <binding name="db-inbound_binding" type="tns:db-inbound_ptt">
    <soap:binding style="document" 
                  transport="http://www.abc.com/wli/sb/transports/jca"/>
    <operation name="receive">
      <soap:operation soapAction="receive"/>
        <input name="receive">
          <soap:body use="literal"/>
        </input>
    </operation>
  </binding>
 
  <service name="db-inbound">
    <port name="db-inbound_pt" binding="tns:db-inbound_binding">
      <soap:address location="jca://eis/DB/OSBJCADBConnection"/>
    </port>
  </service>
</definitions>

The existing JCA WSDL in the config jar will be updated with the concrete WSDL.

JCA Service Upgrade

Each JCA Service in the config jar will be updated with a change in JCA transport specific configuration. A reference to the generated JCA resource will be added to the JCA transport specific configuration section. All other properties defined in the endpoint configuration will remain the same. The JCA service will still contain the same dependency on the same WSDL and binding, although the content of the WSDL is updated to contain the generated concrete WSDL.

TopLink Mapping File Content

TopLink Mapping file content will be extracted from the endpoint configuration. An XML resource will be created the toplink mapping file content extracted from endpoint configuration will be stored as the content for the XML resource. The name of the XML resource will be the toplink mapping file name specified in activation/interaction spec property in endpoint configuration without the “.xml” file extension.A dependency on the generated toplink mapping file XML resource will be generated in the JCA file resource for this JCA service.

AQ Adapter WSDL with Header Upgrade

Prior to SOA 11g, the Header element definition in AQ adapter WSDL is defined with the namespace that is the same as the target namespace of WSDL itself. In SOA 11g, AQ adapter has changed the target namespace for the Header element in WSDL. The new namespace is a fixed namespace as the following:

http://xmlns.oracle.com/pcbpel/adapter/aq/headers/payloadheaders/

OSB JCA transport upgrader will automatically modify the target namespace for the Header element in AQ adapter WSDL to match the new fixed target namespace listed above.

Manual Upgrade

The JCA adapters headers used in 10g is now available in the NormalizedMessage properties. The NormalizedMessage header properties is used in 11g, when AQ adapter needs to have payload header. If payload header is required, then queue header and payload header will be transmitted through NormalizedMessage headers

Due to the change in header support, you must upgrade some Oracle Service Bus 10g configuration manually. Specifically, if the pipeline is accessing message header types that are not present in 11g, the configuration will have to be upgrade to access the header through Transport Header.

3.3.3 JMS Business Service Configuration

JMS business services using the request/response pattern (both MessageID as well as CorrelationID based correlation) will be upgraded to use the new configuration. The existing Is response required flag is enhanced with the Response Queues property option. You can specify None, One per Request URI, or One for all Request URIs as the value for the Response Queues property option. The following table shows how the existing values are upgraded.

Table 3-3 JMS Business Service Configuration

Is Response Required Response Queues

True

One for all Request URIs

False

None


When the Is Response Required value is set to true, then the same value is used in Oracle Service Bus 11g, and the Response Queues option is set as One for all Request URIs.

When the Is Response Required value is set to true in the pre upgrade services then the Response Queues option is set as One for all One for all Request URIs with new configuration Response-URI for each target. In a standalone domain, this target will be a single server. The existing connection-factory and the response JNDI names are merged with the new ResponseURI configuration. The ResponseURI format is created with the available host/port information from the first service URI from business service and appended with the connection-factory and the response queue JNDI name. The host/port information is always retrieved from first service URI. The first service URI also contains a connection-factory, which is retrieved if the configured connection-factory for response is empty.

The format of Response URI is as follows: 'jms://<ip>:<port>,<ip>:<port>/<connection-factory-jndi-name>/<response-queue-jndi-name>'

3.3.4 Global Operational Settings

When you import a Global Operational Settings resource that does not contain result-caching element, then a new result-caching element is added with the default value true.

3.3.5 UDDI Configuration

UDDI related elements have been moved from the ServiceEntry element in Services.xsd to a new element uddiConfiguration. This element is used for business services that are imported from UDDI.

After the upgrade, during synchronization if the WSDL exist with the old name, then this name will be used. However if the WSDL does not exist then a new wsdl will be created with the new naming conventions.

The service key as well as the service name for the service will now be stored as part of the uddiConfiguration element.

3.4 Changes to Carriage Return Handling in XML in 11g Release 1 (11.1.1.6.0) and Later

An update was made to Apache XMLBeans, which corrects handling of carriage return characters in an XML document. The update was made because XMLBeans did not properly escape carriage return characters (\r). If the document contained &#13;, it was correctly unescaped to \r when parsed into XMLBeans; however, \r was not escaped when it was streamed out. This resulted in an invalid XML document because \r is invalid in XML. After the fix to XMLBeans, \r is correctly escaped to &#13;.

If you upgrade to version 11g Release 1 (11.1.1.6.0) or later, and currently have a workaround for the above issue in place, you need to modify how return characters are handled. For example, if the workaround was to escape &#13; a second time to &amp;#13; in order for it to be streamed back out as &#13;, you need to remove the second escape to avoid validation errors.