Oracle Cloud Infrastructure Documentation

Acquiring Kafka Delegation Token for Oozie Workflow

Acquire Kafka delegation tokens for use with an Oozie workflow.

For more information on Kafka delegation tokens, see Using Kafka Delegation Tokens.
  1. Add the credentials of type kafka under the credentials block in your workflow.xml and include the following properties: 
    oozie.kafka.bootstrap.servers  : {Your kafka bootstrap servers}
oozie.kafka.sasl.kerberos.service.name  : kafka
oozie.kafka.security.protocol           : SASL_PLAINTEXT
oozie.kafka.sasl.mechanism              : GSSAPI

    The oozie.kafka.bootstrap.servers: field is a comma-separated list of "host and port" pairs that are the addresses of the Kafka brokers. Specify one or more.

    To pass other properties, add the oozie.kafka. prefix to the property key as shown above. You can pass these either from the Oozie site or workflow.xml.

    Note

    The properties defined in workflow.xml will take higher precedence than the Oozie site.

    Optionally, you can enter the properties from the Ambari. For more information, see Entering Properties Using Ambari.

  2. Include the credentials inside your workflow action: 
    <action name='mr-node' cred='my-kafka-creds'>

    Example:

    <workflow-app xmlns="uri:oozie:workflow:1.0" name="example-wf">
   <credentials>
 
      <credential name="my-kafka-creds" type="kafka">
         <property>
            <name>oozie.kafka.bootstrap.servers</name>
            <value>bootstrapServer1:6667,bootstrapServer2:6667,bootstrapServer3:6667,</value>
         </property>
         <property>
            <name>oozie.kafka.sasl.kerberos.service.name</name>
            <value>kafka</value>
         </property>
         <property>
            <name>oozie.kafka.security.protocol</name>
            <value>SASL_PLAINTEXT</value>
         </property>
         <property>
            <name>oozie.kafka.sasl.mechanism</name>
            <value>GSSAPI</value>
         </property>
      </credential>
 
   </credentials>
 
   <start to="mr-node"/>
   <action name="mr-node" cred='my-kafka-creds'>
      ...
 
   </action>
 
</workflow-app>
  3. Use the text identifier KAFKA_DELEGATION_TOKEN to fetch the Kafka delegation token from the job configuration.
    Note

    • If you are using the AclAuthorizer for Kafka, ensure the correct ACLs are set for users. Add the ACLs on the Kafka broker as follows:

      bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:tokenRequester --allow-host * --operation CreateTokens --user-principal User:tokenOwner

    • If you are using the RangerKafkaAuthorizer for Kafka, ensure the appropriate policies are set. Specifically, add the token requester (in this case Oozie user) under the all - User policy and grant the Create Tokens and Describe Tokens permission. This enables the token requester the permission to acquire a delegation token on behalf of the user who submitted the job.

      The above ACL or Ranger authorization comes into effect only if impersonation is used. For example, token requester != token owner.

    • The Oozie server's Keytab and Principal defined on the Oozie site (oozie.authentication.kerberos.keytab and oozie.authentication.kerberos.principal) are used for authenticating with the Kafka broker.

Entering Properties Using Ambari (Optional)

Enter Kafka configuration properties in Ambari to acquire the Kafka delegation token for an Oozie workflow.

  1. Access Apache Ambari.
  2. From the side toolbar, under Services click Oozie.
  3. Click Configs.
  4. On the Advanced tab, navigate to Custom oozie-site.
  5. Click Add Property and add the following properties:
    • oozie.kafka.bootstrap.servers: Enter your Kafka bootstrap servers in a comma-separated list of "host and port" pairs that are the addresses of the Kafka brokers. Specify one or more.
    • oozie.kafka.sasl.kerberos.service.name: Enter kafka.
    • oozie.kafka.security.protocol: Enter GSSAPI.
    • oozie.kafka.security.protocol: Enter SASL_PLAINTEXT