Webhook Collector

The Webhook Collector microservice receives payloads in HTTP requests from various vendors to a custom endpoint. It converts the payloads to JSON, adding metadata and contextual information, and wraps them in a Unified Assurance JSON wrapper to ensure a standard structure. The transformed payload is then published to the configured Apache Pulsar topic.

This microservice is a supporting microservice that can participate in any microservice pipeline. See Understanding Microservice Pipelines in Unified Assurance Concepts for conceptual information about microservice pipelines.

This microservice provides additional Prometheus monitoring metrics. See Webhook Collector Self-Monitoring Metrics.

This microservice supports the following security mechanisms:

You configure the security mechanism by creating appropriate Kubernetes secrets and specifying the keystore, username, or certificate details in the configuration parameters when you deploy the microservice.

Webhook Collector Prerequisites

Before deploying the microservice, complete the following prerequisite tasks:

  1. Confirm that a microservice cluster is set up. See Microservice Cluster Setup.

  2. Deploy the Pulsar microservice. See Pulsar.

  3. Create the appropriate Kubernetes secrets for your authentication type as the assure1 user:

    • For Basic authentication, create a credentials secret:

      a1k create secret generic webhook-collector-cred-secret --from-literal=authPassword=<basic_auth_password> -n <namespace>
      

      where <namespace> is the namespace where you are deploying the microservice. The default namespace is a1-zone1-pri, but you can change the zone number and, when deploying to a redundant cluster, change pri to sec.

    • For an SSL keystore:

      1. Create a PKCS #12 certificate:

        openssl pkcs12 -export -passout pass:<password> -in <path_to_cert_file> -inkey <path_to_key_file> -out <output_file_name>.p12 -name <name>
        

        where <path_to_p12_cert> is the path to the .p12 certificate that you created.

        Note:

        Because external messages come through an ingress controller before reaching Webhook Collector, SSL/TLS authentication is terminated at the ingress controller level.

      2. Create a certificate secret:

        a1k create secret generic webhook-collector-cert-secret --from-file=<path_to_p12_cert> -n <namespace> 
        
    • For asymmetric JWT (RSA):

      a1k create secret generic webhook-collector-cert-secret --from-file=<cert_file>.crt -n <namespace>
      

      where <cert_file> is the name of the public key certificate file. For example, User-assure1.

    • For shared key JWT (HMAC):

      a1k create secret generic webhook-collector-jwt-secret --from-literal=jwtSecret=<shared_jwt_key> -n <namespace>
      

Deploying Webhook Collector

To deploy the microservice, run the following commands:

su - assure1
export NAMESPACE=<namespace>
export WEBFQDN=<WebFQDN>
a1helm install <microservice-release-name> assure1/webhook-collector -n $NAMESPACE --set global.imageRegistry=$WEBFQDN

In the commands:

You can also use the Unified Assurance UI to deploy microservices. See Deploying a Microservice by Using the UI for more information.

Changing Webhook Collector Configuration Parameters

When running the install command, you can optionally change default configuration parameter values by including them in the command with additional --set-string arguments. You can add as many additional --set-string arguments as you need.

For example, set a parameter described in Default Webhook Collector Configuration by adding --set-string configData.<parameter_name>=<parameter_value>. For example, --set-string configData.LOG_LEVEL=DEBUG.

Default Webhook Collector Configuration

The following table describes the default configuration parameters found in the Helm chart under configData for the microservice.

Name Default Value Possible Values Notes
LOG_LEVEL INFO FATAL, ERROR, WARN, INFO, DEBUG The logging level used by the microservice.
STREAM_OUTPUT "persistent://assure1/event/collection" Text The Pulsar topic to send messages to.
WEBHOOK_ENDPOINT "/webhook" Text The endpoint exposed to receive messages. See Webhook Collector URL Construction.
WEBHOOK_KEYSTORE "" Text Required for TLS/SSL authentication only. The name of the keystore.
KEYSTORE_PASSWORD "" Text Required for TLS/SSL authentication only. The keystore password to use for TLS/SSL authentication.
AUTH_USERNAME "" Text Required for Basic authentication only. The plaintext username.
JWT_CERT "" Text ending with .crt Required for JWT RSA authentication only. The public key file to validate the JWT token. This must use the .crt extension. For example: User-assure1.crt
JWT_ISSUER "" Text For JWT authentication only. Optional. The subject of the token.
JWT_AUDIENCE "" Text For JWT authentication only. Optional. The audience of the token.
JWT_SUBJECT "" Text For JWT authentication only. Optional. The issuer of the token.

Note:

You cannot use both JWT and Basic authentication the same time. If you set parameters for both, JWT is used and Basic is ignored.

Webhook Collector Self-Monitoring Metrics

The Webhook Collector microservice exposes the self-monitoring metrics described in the following table to Prometheus.

Metric Name Type Description
webhook_received_payload_count Counter The total number of payloads received on the webhook.
webhook_to_pulsar_payload_count Counter The total number of payloads sent to Pulsar.
total_errors_occurred Counter The total number of errors occurred in the webhook.

Note:

Metric names in the database include a prefix that indicates the service that inserted them. The prefix is prom_ for metrics inserted by Prometheus. For example, total_errors_occurred is stored as total_errors_occurred in the database.

Webhook Collector URL Construction

The Webhook Collector provides an endpoint that external vendors can publish data to. When deploying the microservice, you can use the default webhook endpoint, or you can set it to a custom value.

The structure of the endpoint URL is:

http://<master_host_of_cluster>:9080/webhook-service/<value_of_configData_WEBHOOK_ENDPOINT>

For example, if you use the default value for WEBHOOK_ENDPOINT, the URL will be:

http://<master_host_of_cluster>:9080/webhook-service/webhook

About Webhook Collector Transformations

The Webhook Collector microservice accepts the following Content-Type for input HTTP request payloads:

When the microservice converts these payloads to JSON, it adds the following fields:

About Custom FCOM Rules for Webhook Events

You primarily use Webhook Collector to send messages to the FCOM Processor, which further normalizes the payloads into the standard Unified Assurance event structure and sends them to the Event Sink microservice for insertion into the Event database. You can optionally create custom FCOM rules to process webhook events in custom ways according to your environment and needs.

The rules must set method to webhook, and can use webhook as a key for an array of additional fields that must appear in the event for the rules to be applied. In the array, set the $.webhook.endpoint key to the value of the WEBHOOK_ENDPOINT configuration parameter. For example, if you use the default endpoint value:

"method": "webhook",
  "webhook": {
    "$.webhook.endpoint": "\/webhook"
  }

The default FCOM rules for the Aruba Central Poller microservice show an example of using the webhook method and key. You can see the rules by expanding the following folders in the Rules UI:

Core Rules (core)/Default read-write branch (default)/processing/event/fcom/_objects/aruba

See FCOM Processor, About FCOM Rules for Custom Events, and Fault for more information.