Note:

• This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
• It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.

Use OCI API Gateway, Functions and Observability to Validate JSON Content and Monitor API Headers and Body

Introduction

When we develop distributed applications, especially in architectures based on microservices, we want components that scale and perform well in their execution. They have very complex architectures, components that execute other components, that execute other components, and so on in an infinite number of endless calls.

Planning how to develop each of them is a huge task. You can expose your microservices built on a Kubernetes cluster through the Oracle Cloud Infrastructure API Gateway (OCI API Gateway). There are a series of facilities, such as performing call authentication and authorization, data validation and call optimization, to name just a few. There is also the possibility of executing calls with OCI Functions with the aim of creating personalized authentication and authorization mechanisms, when existing methods are not sufficient to solve the need.

This tutorial will show how to use the custom mechanism to validate some use cases such as:

• Validate the size of JSON parameter data.

• Validate the maximum number of JSON items.

Despite being a mechanism for authentication and authorization in the OCI API Gateway, it could help with some other needs, such as:

• Capture data from HEADER, query parameters or the body of the REST call.

• Send this data to OCI Observability with the aim of facilitating the debugging of problems, which are often impossible to detect without this information.

  Note: If you are sending data to Observability, consider in your code, as a best practice, use of redaction for HEADER or BODY content, like passwords or sensitive data. Another approach could be to turn on/off your function for debugging purposes.

Objectives

• Configure an API Deployment.

• Develop an OCI Function to capture the HEADER and BODY from the API request.

• Validate a body JSON data.

• Send the HEADER and BODY information to OCI Observability.

Prerequisites

• An operational Oracle Cloud tenant. You can create a free Oracle Cloud account with US$ 300.00 for a month to try this tutorial, see Create a Free Oracle Cloud Account.

• An OCI API Gateway instance created and exposed to the Internet, see Creating Your First API Gateway In The Oracle Cloud.

Task 1: Configure OCI Observability

1. Create a log in your OCI tenancy to ingest the logs from your function. Navigate to Observability and Management and select Logs in the OCI Console.

   

2. Click Create custom log.

   

3. Enter a name in the Custom log name field and choose a proper compartment and Log Group.

   

Note: It is important to capture the OCID of your Log, you will need this for your code.

Task 2: Create an OCI Function to capture the HEADERs and BODY from the API request

To execute the following steps, download code from here function.zip.

• Capture the HEADER and BODY from the API request.

• Validate the body JSON data, there is a method to validate the number of items on each array.

• Send the HEADER and BODY information to OCI Observability.

  import io
  import json
  import logging
  import requests
  import oci
  
  from fdk import response
  from datetime import timedelta
  
  def count_items(dictData):
      counting = 0
      for item in dictData:
          if type(dictData[item]) == list:
              counting += len(dictData[item])
          else:
              if not type(dictData[item]) == str:
                  counting += count_items(dictData[item])
      return counting
  
  def handler(ctx, data: io.BytesIO = None):
      jsonData = "API Error"
      c = 0
      try:
          config = oci.config.from_file("./config","DEFAULT")
          logging = oci.loggingingestion.LoggingClient(config)
          rdata = json.dumps({
              "active": True,
              "context": {
                  "requestheader": data.getvalue().decode('utf-8'),
              },
          })
  
          jsonData = data.getvalue().decode('utf-8')
          # Get the body content from the API request
          body = dict(json.loads(data.getvalue().decode('utf-8')).get("data"))["body"]
          body = dict(json.loads(body))
          # Count the number of items on arrays inside the JSON body
          c = count_items(body)
  
          # If JSON body content has more than 1 item in arrays, block the authorization for the API backend
          if (c > 1):
              # Send a log to observability with out of limit of items in array
              put_logs_response = logging.put_logs(
                  log_id="ocid1.log.oc1.iad.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
                  put_logs_details=oci.loggingingestion.models.PutLogsDetails(
                      specversion="EXAMPLE-specversion-Value",
                      log_entry_batches=[
                          oci.loggingingestion.models.LogEntryBatch(
                              entries=[
                                  oci.loggingingestion.models.LogEntry(
                                      data="out of limit of items in array " + str(c),
                                      id="ocid1.test.oc1..00000001.EXAMPLE-id-Value")],
                              source="EXAMPLE-source-Value",
                              type="EXAMPLE-type-Value")]))
  
              return response.Response(
                  ctx,
                  status_code=401,
                  response_data=json.dumps({"active": False, "wwwAuthenticate": "API Gateway JSON"})
              )
  
          # Send a log to observability with HEADERs and BODY content
          put_logs_response = logging.put_logs(
              log_id="ocid1.log.oc1.iad.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
              put_logs_details=oci.loggingingestion.models.PutLogsDetails(
                  specversion="EXAMPLE-specversion-Value",
                  log_entry_batches=[
                      oci.loggingingestion.models.LogEntryBatch(
                          entries=[
                              oci.loggingingestion.models.LogEntry(
                                  data=jsonData,
                                  id="ocid1.test.oc1..00000001.EXAMPLE-id-Value")],
                          source="EXAMPLE-source-Value",
                          type="EXAMPLE-type-Value")]))
  
          return response.Response(
              ctx, response_data=rdata,
              status_code=200,
              headers={"Content-Type": "application/json"}
          )
      except(Exception) as ex:
          jsonData = 'error parsing json payload: ' + str(ex) + ", " + json.dumps(jsonData)
          pass
  
      return response.Response(
          ctx,
          status_code=401,
          response_data=json.dumps({"active": False, "wwwAuthenticate": jsonData})
      )
  

Understand the Code

This code can be found here function.zip.

Note: If you don’t know how to develop a function and call it in API Gateway, see Call a function using API Gateway.

• count_items method used to count items in any array inside the JSON structure.

  

• This part of code uses the OCI SDK for Python to load the configuration file and the private key to access your OCI tenancy.

  

• rdata will capture the parameters from the request and prepare the response to authorize the function to call the backend of the API Gateway configuration. The active = True will execute the authorization.

  

• jsonData will be used to generate the HEADERs and BODY content to OCI Observability.

  

• This code will capture only the BODY JSON structure from the request.

  

• Below code will count the items on the arrays inside the BODY JSON structure. If the items count overtake more than 1 item, active will be set to False and an error log will be send to OCI Observability. To send the log, you will need to use the Python OCI SDK.

  Note: Replace the log_id variable with your OCID Log generated in Task 1.

  

  If the count is less than or equal to 1, a log with request HEADERs and BODY content will be generated in OCI Observability. Remember to replace the log_id variable with your OCID log.

  Note: You can produce logs in different OCI Log. In this tutorial, only one log was created, however you can create more logs.

  

• In case of an error, a message with the error will be generated here.

  

Configure the SDK Authentication to OCI

You need to configure the config file and put your OCI private key and fingerprint with your function before deploying it to OCI. You must have the config and private key files generated on your Oracle Cloud Infrastructure Command Line Interface (OCI CLI) installation and configuration.

To install and configure your OCI CLI, see Install the OCI CLI. This installation and configuration will generate two files for you. Find config and private key file (default is oci_api_key.pem). The folder path will be informed in the installation instructions.

Download function.zip to see the code, the config file and the private key. Replace the config and private key files with your OCI CLI files.

Build and deploy the OCI Function

In this step, we will need to use the OCI CLI to create the OCI functions and deploy code into your tenancy. To create an OCI function, see OCI Functions QuickStart and search for Python option. You will need to create your function with this information:

• Application: fn_apigw_json

Remember the compartment you deployed your function. You will need this information to configure your OCI API Gateway deployment.

Task 3: Configure the OCI Function in API Gateway

Let’s deploy your API and integrate with your OCI Functions to validate and send request parameters (Header and BODY) to OCI Observability. If you don’t know how to expose your backend in the OCI API Gateway, see OCI API Gateway: Setup, Create and Deploy an API.

1. Open Edit deployment.

   

2. Click Authentication Section.

   

3. Click Single Authentication and select Authorizer Function.

   

4. Choose your functions compartment (where you deployed your function), select fn_apigw_json application and your function python-json-header.

   

5. Configure the Functions Arguments to capture the HEADER and BODY. Capture the HEADER named header and header2, and the BODY content that will be named as body.

   

6. Click Routes and configure the Header Transformation. This configuration is optional, just to see the response content with the request data (HEADER and BODY content) or the errors generated on the request. It will be useful to debug your function.

   

Task 4: Test your Request

Note: In your API Deployment, a cache for the Functions arguments will be activated if you configure the Authorizer Function and set up the Functions Arguments. You can establish what data type will be cached. You can configure cache for query parameter or header but not for body content.

We can test the API request. Let’s test with only one item on an array in the body.

curl --location 'https://xxxxxxxxxxxxxxxxxxxx.apigateway.us-ashburn-1.oci.customer-oci.com/path_index/path' \
    --header 'Content-Type: text/plain' \
    --header 'header: header' \
    --header 'header2: header2' \
    --header 'header3: header3' \
    --data '{"data": {"clientID": "xxxxxxxxxxxxxxxxxxx", "secretID": "xxxxxxxxxxxxxxxxxxx", "jList":[{"added_by":"Ani","description":"example description.","start_date":"2014-10-10","mark":255,"id":975}]}}' -i

The header3 was sent but showed in the log because it was not configured as a Function Argument in OCI API Gateway. There is only 1 item on BODY JSON array, so it is a valid authorization request.

Let’s put one more item on array and test.

curl --location 'https://xxxxxxxxxxxxxxxxxxxx.apigateway.us-ashburn-1.oci.customer-oci.com/path_index/path' \
--header 'Content-Type: text/plain' \
--header 'header: header' \
--header 'header2: header2' \
--header 'header3: header3' \
--data '{"data": {"clientID": "xxxxxxxxxxxxxxxxxxx", "secretID": "xxxxxxxxxxxxxxxxxxx", "jList":[{"added_by":"Ani","description":"example description.","start_date":"2014-10-10","mark":255,"id":975}, {"added_by":"Ani","description":"example description.","start_date":"2014-10-10","mark":255,"id":975}]}}' -i

Related Links

• Overview of OCI API Gateway

• Overview of OCI Functions

• OCI Observability - Observability and Management in the Cloud

• OCI SDK API Reference - LoggingClient

• Python OCI SDK Example

• Creating Your First API Gateway In The Oracle Cloud

• Functions: Call a Function using API Gateway

• Install the OCI CLI

• Functions QuickStart

• OCI API Gateway: Setup, Create and Deploy an API

Acknowledgments

• Author - Cristiano Hoshikawa (Oracle LAD A-Team Solution Engineer)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Use OCI API Gateway, Functions and Observability to Validate JSON Content and Monitor API Headers and Body

F89782-01

November 2023

Copyright © 2023, Oracle and/or its affiliates.