8.2.25 OCI Streaming
Oracle Cloud Infrastructure Streaming (OCI Streaming) supports putting messages to and receiving messages using the Kafka client. Therefore, Oracle GoldenGate for Big Data can be used to publish change data capture operation messages to OCI Streaming.
Note:
The Oracle Streaming Service currently does not have a schema registry to which the Kafka Connect Avro converter can connect. Streams to which the Kafka Handlers or the Kafka Connect Handlers publish messages must be pre-created in Oracle Cloud Infrastructure (OCI). Using the Kafka Handler to publish messages to a stream in OSS which does not already exist results in a runtime exception.- To create a stream in OCI, in the OCI console. select Analytics,
click Streaming, and then click Create Stream. Streams are created by
default in the DefaultPool.
Figure 8-1 Example Image of Stream Creation
- The Kafka Producer client requires certain Kafka producer configuration
properties to connect to OSS streams. To obtain this connectivity information, click
the pool name in the OSS panel. If
DefaultPool
is used, then click DefaultPool in the OSS panel.Figure 8-2 Example OSS Panel showing DefaultPool
Figure 8-3 Example DefaultPool Properties
- The Kafka Producer also requires an AUTH-TOKEN (password) to connect to
OSS. To obtain an
AUTH-TOKEN
go to the User Details page and generate anAUTH-TOKEN
. AUTH-TOKENs are only viewable at creation and are not subsequently viewable. Ensure that you store theAUTH-TOKEN
in a safe place.Figure 8-4 Auth-Tokens
Once you have these configurations, you can publish messages to OSS.
For example, kafka.prm
file:
replicat kafka
TARGETDB LIBFILE libggjava.so SET property=dirprm/kafka.properties
map *.*, target qatarget.*;
kafka.properties
file
:gg.log=log4j gg.log.level=debug gg.report.time=30sec gg.handlerlist=kafkahandler gg.handler.kafkahandler.type=kafka gg.handler.kafkahandler.mode=op gg.handler.kafkahandler.format=json gg.handler.kafkahandler.kafkaProducerConfigFile=oci_kafka.properties # The following dictates how we'll map the workload to the target OSS streams gg.handler.kafkahandler.topicMappingTemplate=OGGBD-191002 gg.handler.kafkahandler.keyMappingTemplate=${tableName} gg.classpath=/home/opc/dependencyDownloader/dependencies/kafka_2.2.0/* jvm.bootoptions=-Xmx512m -Xms32m -Djava.class.path=ggjava/ggjava.jar:dirprm
Example Kafka Producer Properties
(oci_kafka.properties
)
bootstrap.servers=cell-1.streaming.us-phoenix-1.oci.oraclecloud.com:9092 security.protocol=SASL_SSL sasl.mechanism=PLAIN value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="paasdevgg/oracleidentitycloudservice/user.name@oracle.com/ocid1.streampool.oc1.phx.amaaaaaa3p5c3vqa4hfyl7uv465pay4audmoajughhxlsgj7afc2an5u3xaq" password="YOUR-AUTH-TOKEN";
To view the messages, click Load Messages in OSS.
Figure 8-5 Viewing the Messages
![Messages can be viewed in OSS by pressing the “Load Messages” command button. Messages can be viewed in OSS by pressing the “Load Messages” command button.](img/oss_5_load-messages.png)
Parent topic: Target