Oracle Cloud Infrastructure Documentation

Configuring mTLS

Mutual TLS (mTLS) is an extension of the standard Transport Layer Security (TLS) protocol that ensures two-way authentication, adding another layer of security beyond standard TLS. Complete the necessary tasks to configure mTLS authentication for a cluster in Streaming with Apache Kafka.

  1. Generate certificates
  2. Update Kafka cluster with created certificates
  3. Configure Kafka clients

Generating Certificates

In mTLS, both the Kafka client and the Kafka brokers verify each other's identity using digital certificates. Both the Kafka client and broker must have their own certificates and the certificates must be signed by a Certificate Insurance (CA). A CA is a trusted entity that issues, signs, and stores digital certificates.

mTLS uses different types of digital certificates.

  • Client certificates: issued to client (Kafka client) to authenticate a client to a server (Kafka broker) and typically signed by a CA.
  • Server certificates: issued to server (Kafka broker) to authenticate to a client (Kafka client) and typically signed by a CA.
  • Intermediate certificates: sit between the root CA and the leaf certificates (client or server certificates). Intermediate certificates are issued by a CA and can issue leaf certificates. Intermediate certificates enhance security by limiting access to the root CA.
  • Root certificates: are the highest level of certificates in the trust hierarchy and are used to sign intermediate certificates. Root certificates are self-signed. They're installed in a trusted certificate store.

By default, Streaming with Apache Kafka broker certificates are signed by Digicert public root CA with name DigiCert Global Root G2. DigiCert is a trusted CA and its root certificate is likely already included in the default truststore of most JDKs and client tools and you can use it to establish the secure connection.

This is an example of creating certificates using a custom truststore that you can use for test clusters. For production clusters, use certificates signed by a trusted CA.

  1. Ensure you have openssl installed. 
    openssl version
  2. Generate a private key to be used for the root certificate. The following command generates a 4096 bit private key using the RSA algorithm, encrypts it with the AES-256 algorithm using the specified password, and saves it to a file named rootCA.key. 
    openssl genpkey -algorithm RSA -out rootCA.key -aes256 -pass pass:<yourpassword> -pkeyopt rsa_keygen_bits:4096
  3. Create a self-signed root certificate using the private key generated in the step 2. The following command creates a new self-signed X.509 certificate using the private key stored in rootCA.key. The certificate is valid for 10 years, signed with SHA-256, and the output is saved to rootCA.pem. 
    openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 3650 -out rootCA.pem -passin pass:<yourpassword>
  4. Generate a private key to be used for a leaf certificate. The following command generates a 2048 bit private key using the RSA algorithm and saves it to a file named leaf.key. 
    openssl genpkey -algorithm RSA -out leaf.key -pkeyopt rsa_keygen_bits:2048
  5. Create a Certificate Signing Request (CSR) for a leaf certificate using the private key generated in step 4. The following command generates a new CSR using the private key stored in leaf.key and saves the CSR to a file named leaf.csr. This CSR can be sent to a Certificate Authority (CA) to obtain a signed certificate. 
    openssl req -new -key leaf.key -out leaf.csr
  6. Create a signed certificate using the CSR generated in step 5. The following command takes the CSR from leaf.csr, signs it using the root CA rootCA.pem and its private key stored in rootCA.key, and creates a signed certificate saved as leaf.crt. The certificate is valid for 825 days and uses SHA-256 for its signature. The command also generates a serial number for tracking the certificate. 
    openssl x509 -req -in leaf.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out leaf.crt -days 825 -sha256 -passin pass:<yourpassword>

Updating the Kafka Cluster

Update the Kafka cluster with the created certificates.

  1. On the Kafka clusters list page, find the cluster that you want to work with. If you need help finding the list page or the cluster, see Listing Clusters.
  2. From the Actions menu (three dots) for the cluster, select Edit cluster.
  3. In the Security settings section of the Edit cluster panel, enter the certificate content.
  4. Select Update.

Configuring Kafka Clients

To connect to a Kafka cluster using mTLS, after you set up the Kafka truststore, you update the Kafka client properties file.

  1. Run the following command to package the leaf certificate and its private key into a single file named kafka-keystore.p12. 
    openssl pkcs12 -export -in <leaf.crt> -inkey <leaf.key> -out kafka-keystore.p12 -name <kafka-key>
  2. You're prompted to enter a password for the kafka-keystore.p12 file. Remember the password you enter as you need it for the next step along with the location of the file.
  3. Create a client.properties file with the following information: 
    security.protocol=SSL
ssl.certificate.location=<path/to/leaf.cert>
ssl.key.location=<path/to/leaf.key>
ssl.keystore.password=<password-created-in-previous-step>
ssl.keystore.location=<path/to/kafka-keystore.p12>
  4. Change the permissions for the client.properties file. 
    chmod 600 </path/to/client.properties>