Using HashiCorp Vault as Secrets Store

Oracle Health Insurance applications use a secure "secrets store" for the following:

Credentials used to access external web services. For example, these credentials are needed when a call to an external web service is configured to use Basic Authentication.
Key pairs used to establish a trust relation between Oracle Health Insurance applications and an OAuth2 Authorization Server when use of OAuth2 is configured.

Oracle Health Insurance applications support the following secrets stores for credentials and key pairs:

Oracle Platform Security Services (OPSS). This requires use of the JRF template with the WebLogic Infrastructure Installer. This is the default value.
HashiCorp Vault, referred to as Vault from now on. To use Vault as secrets store set system property ohi.secure.secrets.store to value vault (restart required).

This chapter lists requirements and suggestions for installing and using Vault for use with OHI applications. It is not an Installation Guide for Vault, nor a replacement of the Vault documentation. The remainder of this chapter assumes sufficient knowledge of Vault concepts.

It is suggested to set up a central Vault instance for use with multiple OHI applications instances and, possibly, to support other use cases as well.

Required: use Key Value Secrets Engine - Version 2

Oracle Health Insurance applications only support a Vault KV secrets engine version 2.

By default, Oracle Health Insurance applications assume that the KV secrets engine is available at path secret. Enable the KV secrets engine and make it available at path secret with the following Vault CLI command:

vault secrets enable -version=2 -path=secret kv

If the KV secrets engine is made available using a different path, change the Oracle Health Insurance applications' configuration to point to that path by setting the value of system property ohi.vault.kv.secrets.engine accordingly.

Required: define namespaces for storing secrets in Vault

Vault stores secrets in namespaces. Oracle Health Insurance applications build up the namespace to create and access secrets using the following attributes:

By default, Oracle Health Insurance applications assume that Vault’s Key-Value secrets engine is enabled at root path secret.
Under the root path, an Oracle Health Insurance specific namespace section is added. The name is configurable, default value ohi. Creating an Oracle Health Insurance specific section allows the central Vault to be used for other use cases as well. If Vault is configured to use a different Oracle Health Insurance specific namespace section then change system property ohi.vault.namespace accordingly.
A use case specific entry is added to the namespace: credential for credentials and keystore for key stores that contain key pairs.
The (internal) name of the Oracle Health Insurance application is subsequently added to the path, for example, claims for Claims, policies for Policies.
The value of system property ohi.vault.environment.identifier is used to further distinguish secrets on a per Oracle Health Insurance application instance basis. Sample ohi.vault.environment.identifier values might include "configmaster"; "production"; "uat"; etc. Note that a value for this property is required and there is no default.
Finally, a use case specific key is added under which a specific credential or key pair is stored. Customers define key values at the time of configuration.

For example, for a production Claims application, a credential with key "customer_key" would be stored as follows (specified in Vault CLI syntax):

vault write ohi/credential/claims/production/customer_key value=credential

In order for Oracle Health Insurance applications to connect to Vault, the Vault’s address needs to be passed at startup. A valid access token is needed every time the Vault is actually accessed. The following paragraphs list requirements with respect to the address and the token.

Required: encrypt secrets in transit

Vault stores data in an encrypted way. Oracle Health Insurance applications require that data that is exchanged with Vault is also encrypted.

Make sure to run Vault with a proper certificate in place.

If the URI for accessing Vault that is passed to Oracle Health Insurance applications at startup does not start with "https" the Oracle Health Insurance application logs an IllegalStateException and does not communicate with Vault.

The listener section of the Vault configuration should configure the following:

listener "tcp" {
  address = "0.0.0.0:8200"  (1)
  tls_disable = 0  (2)
  tls_cert_file = "/vault/config/cert.pem"  (3)
  tls_key_file = "/vault/config/privkey.pem"
}

1	The port number used here is an example.
2	Make sure not to disable TLS handshakes.
3	See the Vault documentation for configuring certificates.

Also, make sure that any communication between Vault and its configured storage backend is fully encrypted.

Pass Vault’s https-based address to the Oracle Health Insurance application instance by setting it as value for system property ohi.vault.address.

Required: set up Vault Policies and Tokens for different Oracle Health Insurance application instances

Initialization is the process by which Vault’s storage backend is prepared to receive data. During initialization, an "Initial Root Token" is generated.

Avoid passing the root token to Oracle Health Insurance applications.

Instead, it is recommended to create an Oracle Health Insurance application instance specific policy to grant the correct capabilities for a namespace and a matching token for that.

For example, for a development Claims application, the following Vault policy must be created (for example, in a file named "ohi-claims-development-policy.hcl") to support credential management and keystore management:

path "secret/ohi/keystore/claims/dev/*" {
  capabilities = ["read", "list"]
}
path "secret/metadata/ohi/credential/claims/dev/*" {
    capabilities = ["read", "delete", "list"]
}
path "secret/metadata/ohi/keystore/claims/dev/*" {
    capabilities = ["read", "delete", "list"]
}
path "secret/data/ohi/credential/claims/dev/*" {
    capabilities = ["create", "read", "update", "delete", "list"]
}
path "secret/data/ohi/keystore/claims/dev/*" {
    capabilities = ["create", "read", "update", "delete", "list"]
}

Note that the use of metadata and data paths is KV secrets engine specific.

In Vault CLI syntax, the policy would be added to Vault as follows:

vault policy write ohi-claims-development-policy /vault/config/ohi-claims-development-policy.hcl

Subsequently, a token can be created that is attached to the policy. The following example shows Vault CLI syntax for creating a Periodic Service Token that is valid for 24 hours and that the system is able to renew (as explained in the following paragraph):

vault token create -display-name=ohi-claims-development-token -policy=ohi-claims-development-policy -period=24h

Pass the application-specific token to the Oracle Health Insurance application instance by setting it as value for system property ohi.vault.token.

Required: ensure the Vault Token does not expire

The system checks for token expiration every 5 minutes following these steps:

First, it checks the value of system property ohi.vault.token. If it was changed then the system replaces the current token value with the updated value and uses the updated value from that moment on. This mechanism supports refreshing tokens by an external mechanism.
Otherwise, if the value of system property ohi.vault.token was not changed, the system checks if the token expires within the next 6 minutes. If that is the case it will try to renew (the lease associated with) the current token using the duration or increment that is configured for the current token. This assumes the token is a Periodic Service Token. If a Vault related exception occurs, for example because the system cannot get details about the current token, then the system logs the exception and otherwise ignores it.

Recommended: set up Vault in a high available way

To prevent Vault from being overloaded with requests, Oracle Health Insurance applications cache credentials and keys in memory.

Nonetheless, it is good practice to select a Vault storage backend that supports high availability and make sure that Vault itself is clustered. When that is the case, Oracle Health Insurance applications should always access Vault through the clustered address.

manage credential and key store secrets via an OHI applications API. Changing secrets via another Vault API, like the command line interface, results in the cache being out of sync with the data in Vault.

Configuring Vault access via System Properties

To establish a secure connection with Vault, Oracle Health Insurance applications support the following options (evaluated in the order mentioned here):

Order Option Description System Properties

Order	Option	Description	System Properties
1	JKS Truststore	The truststore contains Vault’s server-side certificate.	`ohi.vault.jkstruststore.url`
2	JKS keystore	A reference to a JKS keystore containing Vault’s client-side certificate and private key and the password needed to access it (the password can be null). Use this option in a container-based solution, defining property `ohi.vault.jkskeystore.password` in a container-specific secrets location.	`ohi.vault.jkskeystore.url` and `ohi.vault.jkskeystore.password`
3	X.509 certificate	A URL that references the location of an X.509 certificate in unencrypted PEM format, using UTF-8 encoding.	`ohi.vault.pem.url`
4	X.509 client certificate	The URL to an X.509 client certificate in unencrypted PEM format, using UTF-8 encoding combined with the URL to an RSA private key in unencrypted PEM format, using UTF-8 encoding. Both values must be provided.	`ohi.vault.clientpem.url` and `ohi.vault.clientkeypem.url`

JKS Truststore

The truststore contains Vault’s server-side certificate.

ohi.vault.jkstruststore.url

JKS keystore

A reference to a JKS keystore containing Vault’s client-side certificate and private key and the password needed to access it (the password can be null). Use this option in a container-based solution, defining property ohi.vault.jkskeystore.password in a container-specific secrets location.

ohi.vault.jkskeystore.url and ohi.vault.jkskeystore.password

X.509 certificate

A URL that references the location of an X.509 certificate in unencrypted PEM format, using UTF-8 encoding.

ohi.vault.pem.url

X.509 client certificate

The URL to an X.509 client certificate in unencrypted PEM format, using UTF-8 encoding combined with the URL to an RSA private key in unencrypted PEM format, using UTF-8 encoding. Both values must be provided.

ohi.vault.clientpem.url and ohi.vault.clientkeypem.url