35 OCI XStream Functions
This chapter describes the XStream functions for OCI.
A row logical change record (LCR) is used to encapsulate each row change. It includes the schema name, table name, DML operation, and the column values. For update operations, both before and after column values are included. The column data is in the format specified by the "Program Variable" column in Table 35-3. Character columns are converted to the client's character set.
A DDL LCR is used to encapsulate each DDL change. It includes the object name, the DDL text, and the DDL command, for example, ALTER TABLE or TRUNCATE TABLE. See Table A-1 for a list of DDL command codes.
See Also:
Oracle Database Globalization Support Guide for more information about NLS settings.
XStream sample programs are found in xstream/oci under the $ORACLE_HOME/demo directory.
Each LCR also has a transaction ID and position. For transactions captured outside Oracle databases, any byte-comparable RAW array can be used as the LCR position, if the position of each LCR in the stream is strictly increasing.
This chapter contains the topic:
35.1 About Using the XStream Interface
Since Oracle Database 11g Release 2, Oracle Streams provides enhanced APIs, known as eXtended Streams (XStream) Out and XStream In, to enable high performance, near real-time information-sharing infrastructure between Oracle databases and non-Oracle databases, non-RDBMS Oracle products, file systems, third party software applications, and so on.
XStream is built on top of Streams infrastructure.
See Also:
35.1.1 XStream Out
XStream Out allows a remote client to attach to an outbound server (a Streams apply process) and extract row changes in the form of Logical Change Records (LCRs).
To use XStream Out, a capture and an apply process must be created similar to other Streams setup. All data types supported by Oracle Streams including LOB, LONG, and XMLType are supported by XStreams. Such an apply process is called an outbound server. The capture and the outbound server may or may not be on the same database instance. After the capture and the outbound server have started, row changes will be captured and sent to the outbound server. An external client can then connect to this outbound server using OCI. After the connection is established, the client can loop waiting for LCRs from the outbound server. The client can register a client-side callback to be invoked each time an LCR is received. At anytime, the client can detach from the outbound server as needed. Upon restart, the outbound server knows where in the redo stream to start streaming LCRs to the client.
See Also:
-
Oracle Database XStream Guide for more details of XStreams concepts
35.1.1.1 LCR Streams
Describes characteristics of an LCR stream.
-
An LCR stream must be repeatable.
-
An LCR stream must contain a list of assembled and committed transactions.
-
LCRs from one transaction are contiguous. There is no interleaving of transactions in the LCR stream.
-
Each transaction within an LCR stream must have an ordered list of LCRs and a transaction ID.
-
The last LCR in each transaction must be a commit LCR.
-
Each LCR must have a unique position.
-
The position of all LCRs within a single transaction and across transactions must be strictly increasing.
35.1.1.2 The Processed Low Position and Restart Considerations
If the outbound server or the client aborts abnormally, the connection between the two is automatically broken. The client needs to maintain the processed low position to properly recover after a restart.
The processed low position is a position below which all LCRs have been processed by the client. This position should be maintained by the client while applying each transaction. Periodically this position is sent to the server while the client executes XStream Out APIs. This position indicates to the server that the client has processed all LCRs below or equal to this position; thus, the server can purge redo logs that are no longer needed.
Upon restart, the client must re-attach to the outbound server. During the attach call, the client can notify the outbound server of the last position received by the client. The outbound server then sends LCRs with position greater than this last position. If the client does not specify the last position (that is, a NULL is specified), the outbound server will retrieve the processed low position from its system tables and derive the starting position to mine the redo logs. It will send to the client the LCRs with position greater than this processed low position.
35.1.2 XStream In
To replicate non-Oracle data into Oracle databases use XStream In which allows a remote client to attach to an inbound server (a Streams apply process) and send row and DDL changes in the form of LCRs.
An external client application connects to the inbound server using OCI. After the connection is established, the client application acts as the capture agent for the inbound server by streaming LCRs to it. A client application can attach to only one inbound server per database connection. Each inbound server only allows one client attaching to it.
XStream In uses the following features of Oracle Streams:
-
High performance processing of DML changes using an apply process and, optionally, apply process parallelism.
-
Apply process features such as SQL generation, conflict detection and resolution, error handling, and customized processing with apply handlers.
-
Streaming network transmission of information with minimal network round trips.
XStream In supports all of the data types that are supported by Oracle Streams, including LOBs, LONG, LONG RAW, and XMLType. A client application sends LOB and XMLType data to the inbound server in chunks. Several chunks make up a single column value of LOB or XMLType.
35.1.2.1 Processed Low Position and Restart Ability
The processed low position is the position below which the inbound server no longer requires any LCRs.
This position corresponds with the oldest SCN for an Oracle Streams apply process that applies changes captured by a capture process.
The processed low position indicates that the LCRs less than or equal to this position have been processed by the inbound server. If the client re-attaches to the inbound server, it only needs to send LCRs greater than the processed low position because the inbound server discards any LCRs that are less than or equal to the processed low position.
If the client application aborts abnormally, then the connection between the client application and the inbound server is automatically broken. Upon restart, the client application retrieves the processed low position from the inbound server and instructs its capture agent to retrieve changes starting from this processed low position.
To limit the recovery time of a client application using the XStream In interface, the client application can send activity, such as empty transactions, periodically to the inbound server. When there are no LCRs to send to the server, the client can send a row LCR with a commit directive to advance the inbound server's processed low position. This activity acts as an acknowledgement so that the inbound server's processed low position can be advanced. The LCR stream sent to an inbound server must follow the LCR stream properties for XStream Out defined above.
35.1.2.2 Stream Position
Stream position refers to the position of an LCR in a given LCR stream.
For transactions captured outside Oracle databases the stream position must be encoded in certain format (for example, base-16 encoding) that supports byte comparison. The stream position is key to the total order of transaction stream sent by clients using the XStream In interface.
35.1.3 Security of XStreams
Describes the security characteristics of XStreams.
XStream Out allows regular users to receive LCRs without requiring system level privileges. System level privileges, such as DBA role, are required to configure XStream Out. The user who configures XStream Out can specify a regular user as the connect user who can attach to an outbound server to receive LCRs.
XStream In allows regular users to update tables in its own schema without requiring system level privileges (for example, DBA) to configure XStream In.
XStream cannot assume that the connected user to the inbound or outbound server is trusted.
OCI clients must connect to an Oracle database prior to attaching to an XStream outbound or inbound server created on that database. The connected user must be the same as the connect_user configured for the attached outbound server or the apply_user configured for the attached inbound server; otherwise, an error is raised.
See Also:
Oracle Database XStream Guide for more about configuring Oracle XStreams