Configuring the TimesTen Exporter and Prometheus with Client Certificate Authentication

This example illustrates how to configure both the Exporter and Prometheus to require client certificate authentication. It first uses the ttExporter utility to create and export the server certificate, to create and export the client certificate, and to create and export the client private key. It then configures Prometheus to use the server certificate, the client certificate, and the client private key that was created by the TimesTen ttExporter utility. The example starts the Exporter. After the Exporter is started, the example verifies that Prometheus is securely scraping the TimesTen metrics that the Exporter exposes.

Create a subdirectory to store the Oracle Wallet containing the certificate information used by the Exporter. This example creates the mycertdir subdirectory.
```
mkdir -p mycertdir
```
Use the ttExporter utility with the -create-server-certificate to create the self-signed certificate and store an Oracle Wallet containing the certificate information. Use the -certificate-directory option to specify the location for the certificate information (mycertdir, in this example). If you do not specify -certificate-directory, ttExporter creates and stores an Oracle Wallet containing the certificate information in your user's $HOME directory. Perform this step one time.
```
% ttExporter -create-server-certificate -certificate-directory mycertdir
% ls -a mycertdir
.  ..  .ttwallet.C76FCD9A-E5C8-4AC5-A73D-7DAB72203E43
% ls -a mycertdir/.ttwallet.C76FCD9A-E5C8-4AC5-A73D-7DAB72203E43
.  ..  cwallet.sso
```
Use the ttExporter utility with the -export-server-certificate option to export the server certificate in PEM format. This example exports the server certificate to the server.crt file in the mycertdir directory. You must provide the -certificate-directory option since you specified it when you created the server certificate. This example also verifies that the mycertdir/server.crt file was created. Note that you will later supply the mycertdir/server.crt file for the ca_file parameter in the tls_config block of the Prometheus configuration file.
```
% ttExporter -export-server-certificate mycertdir/server.crt -certificate-directory mycertdir
% cat mycertdir/server.crt
-----BEGIN CERTIFICATE-----
MIIDtzCCAp+gAwIBAgIJANV4RZyydnFrMA0GCSqGSIb3DQEBCwUAMHIxCzAJBgNV
...
4PtJlBCXt6ogsceJI8xrCxcfUjPh9TxBWQfeW2vI+AfmUEzXTrYa+xSmkg==
-----END CERTIFICATE-----
```
Use the ttExporter utility with the -export-client-certificate and the -export-client-private-key options to create and export a client certificate to a file (mycertdir/client.crt, in this example) and a client private key to a second file (mycertdir/key.crt, in this example). You must also provide the -certificate-directory option, since you specified it when you created the server certificate. These options must be specified together. This steps also verifies that the mycertdir/client.crt and the mycertdir/key.crt were created. Note that you will later supply the mycertdir/client.crt file for the cert_file parameter and the mycertdir/key.crt file for the key_file in the tls_config block of the Prometheus configuration file.
Note:
Create a client certificate and private key for each Prometheus instance that scrapes metrics from the Exporter.
```
% ttExporter -export-client-certificate mycertdir/client.crt 
    -export-client-private-key mycertdir/key.crt -certificate-directory mycertdir
% cat mycertdir/client.crt
-----BEGIN CERTIFICATE-----
MIIDXTCCAkUCCCPYOo/eJJFMMA0GCSqGSIb3DQEBCwUAMHIxCzAJBgNVBAYTAlVT
...
c18fFPkKcsJIJqYudoY8u1mIzThdjVhA8zY25vPU6exxAKw1BmJPuwqexyIpbNNH
LA==
-----END CERTIFICATE-----
% cat mycertdir/key.crt
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA0/+ERz/MgA34Uv+UNnlRPDnssl4QKyevd05vc/1wrSDwEZJy
...
l+Mi8MgmjKInXkmRotxWbJ6LSQY5wiol5HhwLTTCQgoCdVTO+usaXQ==
-----END RSA PRIVATE KEY-----
```
Configure Prometheus to securely scrape metrics from the Exporter. This requires that you modify the Prometheus configuration file (prometheus.yml) by setting the ca_file, the cert_file, and the key_file parameters in the tls_config block of the scrape configuration (scrape_configs) block. You also must set general parameter settings in the scrape_configs block.
Specifically, modify these parameters in the scrape_configs block:
- job_name: Specify the job name assigned to scrape the metrics (tt_12345, in this example).
- metrics_path: There is no need to specify the metrics_path as the default is /metrics. The Exporter uses the /metrics default as the HTTPS resource path on which Prometheus fetches the TimesTen metrics from the Exporter.
- scheme: You must specify the https scheme to ensure The Exporter and the Prometheus instance operate with client server authentication.
- tls_config block:
  - ca_file: Specify the server certificate file that was exported when you ran ttExporter -export-server-certificate (mycertdir/server.crt, in this example).
  - cert_file: Specify the client certificate file that was created and exported when you ran ttExporter -export-client-certificate (mycertdir/client.crt, in this example)
  - key_file: Specify the client private key that was created and exported when you ran ttExporter - export-client-private-key ( mycertdir/key.crt, in this example)
- static_configs block: For the targets parameter, specify the host that the Exporter will be running on and the port number that the Exporter will be listening on (myhost:12345, in this example).
```
vi prometheus.yml
...
scrape_configs:
  - job_name: 'tt_12345'
  # metrics_path defaults to '/metrics'
  # scheme defaults to 'http'.
  scheme: https
  tls_config:
    ca_file: mycertdir/server.crt
    cert_file: mycertdir/client.crt
    key_file: mycertdir/key.crt
  static_configs:
    - targets: ['myhost:12345']
```
Use the ttExporter utility to start the Exporter. You must specify the -port option. Ensure to specify the same port as what was specified in the Prometheus configuration file (12345, in this example). You must also provide the -certificate-directory option since this option was specified when you create the server certificate. Optionally, specify the -pid-file option to store the process ID of ttExporter in a file. In this example, the file is /tmp/ttexporter.pid and the process ID is 24302.
```
% ttExporter -port 12345 -pid-file /tmp/ttexporter.pid -certificate-directory mycertdir
% cat /tmp/ttexporter.pid
24302
% ps 24302
  PID TTY      STAT   TIME COMMAND
24302 pts/0    Sl+    0:00 timesten_home/bin/_ttExporter -port 12345 -pid-file /tmp/ttexporter.pid 
-certificate-directory mycertdir
```

Optional and for verification purposes only: Navigate to the certificate directory (mycertdir, in this example). Then use the Linux curl command to verify the TimesTen metrics are exposed and available for Prometheus to scrape.

% cd mycertdir
% curl --cacert server.crt --cert ./client.crt --key ./key.crt https://myhost:12345/metrics
...
# HELP timesten_perm_in_use_bytes Bytes of permanent space used
# TYPE timesten_perm_in_use_bytes gauge
timesten_perm_in_use_bytes{dsn="mydsn",instancename="myinstance"} 20385792
# HELP timesten_perm_in_use_high_water_bytes Maximum bytes of permanent space used
# TYPE timesten_perm_in_use_high_water_bytes gauge
timesten_perm_in_use_high_water_bytes{dsn="mydsn",instancename="myinstance"} 20385792
...

Optional: This step illustrates that you can stop the Exporter. This terminates the ttExporter process. Prometheus no longer scrapes the endpoint. Recall that the -pid-file option was specified when you started the Exporter. Use the Linux pkill -F command to terminate the process ID. Recall the example stores the process ID in the /tmp/ttexporter.pid file and that the process ID is 24302.
```
% pkill -F /tmp/ttexporter.pid
% cat /tmp/ttexporter.pid
24302
% ps 24302
  PID TTY      STAT   TIME COMMAND
```

You successfully configured the Exporter and Prometheus to require client certificate authentication. You also started the Exporter. Prometheus is securely scraping the TimesTen metrics that the Exporter exposes at the /metrics endpoint. Prometheus continues to scrape the TimesTen metrics until you stop the Exporter.