Webhook Collector
The Webhook Collector microservice receives payloads in HTTP requests from various vendors to a custom endpoint. It converts the payloads to JSON, adding metadata and contextual information, and wraps them in a Unified Assurance JSON wrapper to ensure a standard structure. The transformed payload is then published to the configured Apache Pulsar topic.
This microservice is a supporting microservice that can participate in any microservice pipeline. See Understanding Microservice Pipelines in Unified Assurance Concepts for conceptual information about microservice pipelines.
This microservice provides additional Prometheus monitoring metrics. See Webhook Collector Self-Monitoring Metrics.
This microservice supports the following security mechanisms:
-
SSL/TLS authentication with a keystore
-
Basic Authentication with username and password validation
-
JWT authentication with shared key validation (HMAC)
-
JWT authentication with asymmetric key validation (RSA)
You configure the security mechanism by creating appropriate Kubernetes secrets and specifying the keystore, username, or certificate details in the configuration parameters when you deploy the microservice.
Webhook Collector Prerequisites
Before deploying the microservice, complete the following prerequisite tasks:
-
Confirm that a microservice cluster is set up. See Microservice Cluster Setup.
-
Deploy the Pulsar microservice. See Pulsar.
-
Create the appropriate Kubernetes secrets for your authentication type as the assure1 user:
-
For Basic authentication, create a credentials secret:
a1k create secret generic webhook-collector-cred-secret --from-literal=authPassword=<basic_auth_password> -n <namespace>where <namespace> is the namespace where you are deploying the microservice. The default namespace is a1-zone1-pri, but you can change the zone number and, when deploying to a redundant cluster, change pri to sec.
-
For an SSL keystore:
-
Create a PKCS #12 certificate:
openssl pkcs12 -export -passout pass:<password> -in <path_to_cert_file> -inkey <path_to_key_file> -out <output_file_name>.p12 -name <name>where <path_to_p12_cert> is the path to the .p12 certificate that you created.
Note:
Because external messages come through an ingress controller before reaching Webhook Collector, SSL/TLS authentication is terminated at the ingress controller level.
-
Create a certificate secret:
a1k create secret generic webhook-collector-cert-secret --from-file=<path_to_p12_cert> -n <namespace>
-
-
For asymmetric JWT (RSA):
a1k create secret generic webhook-collector-cert-secret --from-file=<cert_file>.crt -n <namespace>where <cert_file> is the name of the public key certificate file. For example, User-assure1.
-
For shared key JWT (HMAC):
a1k create secret generic webhook-collector-jwt-secret --from-literal=jwtSecret=<shared_jwt_key> -n <namespace>
-
Deploying Webhook Collector
To deploy the microservice, run the following commands:
su - assure1
export NAMESPACE=<namespace>
export WEBFQDN=<WebFQDN>
a1helm install <microservice-release-name> assure1/webhook-collector -n $NAMESPACE --set global.imageRegistry=$WEBFQDN
In the commands:
-
<namespace> is the namespace where you are deploying the microservice. The default namespace is a1-zone1-pri, but you can change the zone number and, when deploying to a redundant cluster, change pri to sec.
-
<WebFQDN> is the fully-qualified domain name of the primary presentation server for the cluster.
-
<microservice-release-name> is the name to use for the microservice instance. Oracle recommends using the microservice name (webhook-collector) unless you are deploying multiple instances of the microservice to the same cluster.
You can also use the Unified Assurance UI to deploy microservices. See Deploying a Microservice by Using the UI for more information.
Changing Webhook Collector Configuration Parameters
When running the install command, you can optionally change default configuration parameter values by including them in the command with additional --set-string arguments. You can add as many additional --set-string arguments as you need.
For example, set a parameter described in Default Webhook Collector Configuration by adding --set-string configData.<parameter_name>=<parameter_value>. For example, --set-string configData.LOG_LEVEL=DEBUG.
Default Webhook Collector Configuration
The following table describes the default configuration parameters found in the Helm chart under configData for the microservice.
| Name | Default Value | Possible Values | Notes |
|---|---|---|---|
| LOG_LEVEL | INFO | FATAL, ERROR, WARN, INFO, DEBUG | The logging level used by the microservice. |
| STREAM_OUTPUT | "persistent://assure1/event/collection" | Text | The Pulsar topic to send messages to. |
| WEBHOOK_ENDPOINT | "/webhook" | Text | The endpoint exposed to receive messages. See Webhook Collector URL Construction. |
| WEBHOOK_KEYSTORE | "" | Text | Required for TLS/SSL authentication only. The name of the keystore. |
| KEYSTORE_PASSWORD | "" | Text | Required for TLS/SSL authentication only. The keystore password to use for TLS/SSL authentication. |
| AUTH_USERNAME | "" | Text | Required for Basic authentication only. The plaintext username. |
| JWT_CERT | "" | Text ending with .crt | Required for JWT RSA authentication only. The public key file to validate the JWT token. This must use the .crt extension. For example: User-assure1.crt |
| JWT_ISSUER | "" | Text | For JWT authentication only. Optional. The subject of the token. |
| JWT_AUDIENCE | "" | Text | For JWT authentication only. Optional. The audience of the token. |
| JWT_SUBJECT | "" | Text | For JWT authentication only. Optional. The issuer of the token. |
Note:
You cannot use both JWT and Basic authentication the same time. If you set parameters for both, JWT is used and Basic is ignored.
Webhook Collector Self-Monitoring Metrics
The Webhook Collector microservice exposes the self-monitoring metrics described in the following table to Prometheus.
| Metric Name | Type | Description |
|---|---|---|
| webhook_received_payload_count | Counter | The total number of payloads received on the webhook. |
| webhook_to_pulsar_payload_count | Counter | The total number of payloads sent to Pulsar. |
| total_errors_occurred | Counter | The total number of errors occurred in the webhook. |
Note:
Metric names in the database include a prefix that indicates the service that inserted them. The prefix is prom_ for metrics inserted by Prometheus. For example, total_errors_occurred is stored as total_errors_occurred in the database.
Webhook Collector URL Construction
The Webhook Collector provides an endpoint that external vendors can publish data to. When deploying the microservice, you can use the default webhook endpoint, or you can set it to a custom value.
The structure of the endpoint URL is:
http://<master_host_of_cluster>:9080/webhook-service/<value_of_configData_WEBHOOK_ENDPOINT>
For example, if you use the default value for WEBHOOK_ENDPOINT, the URL will be:
http://<master_host_of_cluster>:9080/webhook-service/webhook
About Webhook Collector Transformations
The Webhook Collector microservice accepts the following Content-Type for input HTTP request payloads:
-
application/json
-
application/xml
-
application/x-www-form-urlencoded
-
text/plain
-
application/x-yaml
When the microservice converts these payloads to JSON, it adds the following fields:
-
domain: The event domain. For webhook events, this is fault.
-
@timestamp: The time the message was processed by the Webhook Collector.
-
webhook: The input payload content and headers.
-
headers: The request REST headers and details about the endpoint and query parameters.
-
endpoint: The endpoint to which the request was made, prefixed by the configured endpoint path.
-
queryParams: Any request query parameters.
-
ZoneId: The zone ID associated with the namespace where the event originated.
-
content: The request payload, converted to JSON format. If JSON conversion is not possible, the raw form is shown.
-
rawContent: The payload exactly as it was received, in its original raw format.
-
-
_type: The type of event. For webhook events, this field is set to webhook.
-
source: The IP and DNS details of the message's origin.
-
agents: The details of the Webhook Collector instance that collected the message, useful for backtracking and auditing.
-
_version: The version of the Unified Assurance wrapper.
About Custom FCOM Rules for Webhook Events
You primarily use Webhook Collector to send messages to the FCOM Processor, which further normalizes the payloads into the standard Unified Assurance event structure and sends them to the Event Sink microservice for insertion into the Event database. You can optionally create custom FCOM rules to process webhook events in custom ways according to your environment and needs.
The rules must set method to webhook, and can use webhook as a key for an array of additional fields that must appear in the event for the rules to be applied. In the array, set the $.webhook.endpoint key to the value of the WEBHOOK_ENDPOINT configuration parameter. For example, if you use the default endpoint value:
"method": "webhook",
"webhook": {
"$.webhook.endpoint": "\/webhook"
}
The default FCOM rules for the Aruba Central Poller microservice show an example of using the webhook method and key. You can see the rules by expanding the following folders in the Rules UI:
Core Rules (core)/Default read-write branch (default)/processing/event/fcom/_objects/aruba
See FCOM Processor, About FCOM Rules for Custom Events, and Fault for more information.