Adding Logging to API Deployments

Having created an API gateway and deployed one or more APIs on it, there will likely be occasions when you'll need to see more detail about the flow of traffic into and out of the API gateway. For example, you might want to review responses returned to API clients, or to troubleshoot errors. You can specify that the API Gateway service stores information about requests and responses going through an API gateway, and information about processing within an API gateway, as logs in the Oracle Cloud Infrastructure Logging service.

You can define and store two kinds of logs for API deployments in the Oracle Cloud Infrastructure Logging service:
  • Access logs, that record a summary of every request and response that goes through the API gateway to and from an API deployment. For more information about access log content, see API Deployment Access Log.
  • Execution logs, that record information about processing within the API gateway for an API deployment. For more information about execution log content, see API Deployment Execution Log. You can specify a log level for execution logs as one of the following:
    • Information, to record a summary of every processing stage.
    • Warning, to record only transient errors that occur during processing. For example, a connection reset.
    • Error, to record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.

    You can set an execution log level for an API deployment, and also set different execution log levels for individual routes to override the execution log level inherited from the API deployment.

Note

In earlier API Gateway releases (prior to the release of the Oracle Cloud Infrastructure Logging service), you could direct API Gateway to store access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage. This functionality is still available for any API deployments you created in earlier releases where you previously specified a logging policy.

You can set up a logging policy in the API deployment specification for all routes in the API deployment specification, and optionally also set up different logging policies for individual routes to override the policy set at the API deployment level. Each log object contains the log messages output in a ten minute logging window. Log objects are available in Object Storage approximately ten minutes after the end of the logging window.

However, note the following:
  • If you set up Oracle Cloud Infrastructure Logging to store logs for an API deployment, you are no longer able to store logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage for that API deployment.
  • The ability to store access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage will be deprecated in a future release.
  • To store access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage, you have to define the logging policy in a JSON file. You cannot use the Console to store access logs and execution logs in Oracle Cloud Infrastructure Object Storage.

You can add logging to an API deployment specification by:

  • using the Console
  • editing a JSON file

Using the Console to Add Logging

Using the Console to Configure and Enable Logs in Oracle Cloud Infrastructure Logging

To configure and enable API deployment logs using the Console to store logs in Oracle Cloud Infrastructure Logging:

  1. Create or update an API deployment using the Console, select the From Scratch option, and enter details on the Basic Information page.

    For more information, see Deploying an API on an API Gateway by Creating an API Deployment and Updating API Gateways and API Deployments.

  2. In the API Logging Policies section of the Basic Information page, specify one of the following options as the Execution Log Level to record information about processing within the API gateway:

    • Information: Record a summary of every processing stage. This is the default option.
    • Warning: Record only transient errors that occur during processing. For example, a connection reset.
    • Error: Record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.
  3. Click Next to enter details for individual routes in the API deployment on the Routes page and click Show Route Logging Policies.

  4. Specify one of the following options as the Execution Log Level Override that applies to an individual route (to override the execution log level inherited from the API deployment):

    • Information: Record a summary of every processing stage.
    • Warning: Record only transient errors that occur during processing. For example, a connection reset.
    • Error: Record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.
  5. Click Next to review the details you entered for the API deployment.
  6. Click Create or Save Changes to create or update the API deployment.

    The API deployment is shown on the API Deployment Details page.

  7. Under Resources, click Logs, and then click the Enable Logging slider to create and enable a new API deployment log in the Oracle Cloud Infrastructure Logging service in the Create Log entry panel:
    • Compartment: By default, the current compartment.
    • Log Group: By default, the first log group in the compartment.
    • Log Category: Select either Execution or Access.
    • Log Name: By default, <deployment-name>_execution or <deployment-name>_access, depending on which category you select.

    For more information, see Enabling Logging for a Resource.

  8. Click Enable Log to create the new log and enable it.

Editing a JSON File to Add Logging

Editing a JSON File to Set Execution Log Level for Logs Stored in Oracle Cloud Infrastructure Logging

To edit the API deployment specification in a JSON file to set the log level for execution logs stored in Oracle Cloud Infrastructure Logging:

  1. Using your preferred JSON editor, edit the existing API deployment specification in which you want to set the log level for execution logs stored in Oracle Cloud Infrastructure Logging, or create a new API deployment specification (see Creating an API Deployment Specification).

    At a minimum, the API deployment specification will include a routes section containing:

    • A path. For example, /hello
    • One or more methods. For example, GET
    • A definition of a back end. For example, a URL, or the OCID of a function in Oracle Functions.

    For example, the following basic API deployment specification defines a simple Hello World serverless function in Oracle Functions as a single back end:

    {
      "routes": [
        {
          "path": "/hello",
          "methods": ["GET"],
          "backend": {
            "type": "ORACLE_FUNCTIONS_BACKEND",
            "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
          }
        }
      ]
    }
  2. (Optional) To set the log level for execution logs that applies globally to all routes in the API deployment specification:

    1. Insert a loggingPolicies section before the routes section. For example:

      
      {
        "loggingPolicies": {},
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            }
          }
        ]
      }
    2. Specify the level of detail to record about processing within the API gateway for all routes by including the executionLog policy in the loggingPolicies section, and setting the logLevel property to one of the following:

      • INFO to record a summary of every processing stage.
      • WARN to record only transient errors that occur during processing. For example, a connection reset.
      • ERROR to record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.

      For example:

      
      {
        "loggingPolicies": {
          "executionLog": {
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            }
          }
        ]
      }
  3. (Optional) To set the log level for execution logs for a particular route (overriding the global execution log level inherited from the API deployment):

    1. Insert a loggingPolicies section after the route's backend section. For example:

      
      {
        "loggingPolicies": {
          "executionLog": {
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            },
            "loggingPolicies": {}
          }
        ]
      }
    2. Specify the level of detail to record about processing within the API gateway for the route by including the executionLog policy in the loggingPolicies section, and setting the logLevel property to one of the following:

      • INFO to record a summary of every processing stage.
      • WARN to record only transient errors that occur during processing. For example, a connection reset.
      • ERROR to record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.

      For example:

      
      {
        "loggingPolicies": {
          "executionLog": {
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            },
            "loggingPolicies": {
              "executionLog": {
                "logLevel": "ERROR"
              }
            }
          }
        ]
      }
  4. Save the JSON file containing the API deployment specification.

  5. Use the API deployment specification when you create or update an API deployment in the following ways:

    • by specifying the JSON file in the Console when you select the Upload an existing API option
    • by specifying the JSON file in a request to the API Gateway REST API

    For more information, see Deploying an API on an API Gateway by Creating an API Deployment.

  6. Having set the log level for execution logs, follow the instructions in Enabling Logging for a Resource to create and enable a new API deployment log in the Oracle Cloud Infrastructure Logging service.

Editing a JSON File to Set Up Logging Policies to Store Logs in Object Storage

In earlier API Gateway releases (prior to the release of the Oracle Cloud Infrastructure Logging service), you could direct API Gateway to store and view access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage. This functionality is still available for any API deployments you created in earlier releases where you previously specified a logging policy, as described below. However, note the following:
  • If you set up Oracle Cloud Infrastructure Logging to store logs for an API deployment, you are no longer able to store logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage for that API deployment.
  • The ability to store access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage will be deprecated in a future release.

To edit the API deployment specification in a JSON file to store logs in Object Storage:

  1. Using your preferred JSON editor, edit the existing API deployment specification to which you want to add a logging policy, or create a new API deployment specification (see Creating an API Deployment Specification).

    At a minimum, the API deployment specification will include a routes section containing:

    • A path. For example, /hello
    • One or more methods. For example, GET
    • A definition of a back end. For example, a URL, or the OCID of a function in Oracle Functions.

    For example, the following basic API deployment specification defines a simple Hello World serverless function in Oracle Functions as a single back end:

    {
      "routes": [
        {
          "path": "/hello",
          "methods": ["GET"],
          "backend": {
            "type": "ORACLE_FUNCTIONS_BACKEND",
            "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
          }
        }
      ]
    }
  2. (Optional) To add a logging policy to the API deployment specification that applies globally to all routes in the API deployment specification:

    1. Insert a loggingPolicies section before the routes section. For example:

      
      {
        "loggingPolicies": {},
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            }
          }
        ]
      }
    2. (Optional) Record a summary of every request and response that goes through the gateway by including the accessLog policy in the loggingPolicies section, and setting the isEnabled property to true.

      For example:

      
      {
        "loggingPolicies": {
          "accessLog": {
            "isEnabled": true 
          },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            }
          }
        ]
      }
    3. (Optional) Record information about processing within the API gateway by including the executionLog policy in the loggingPolicies section, setting the isEnabled property to true, and setting the logLevel property to one of the following:

      • INFO to record a summary of every processing stage.
      • WARN to record only transient errors that occur during processing. For example, a connection reset.
      • ERROR to record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.

      For example:

      
      {
        "loggingPolicies": {
          "accessLog": {
            "isEnabled": true 
          },
          "executionLog": {
            "isEnabled": true,
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            }
          }
        ]
      }
  3. (Optional) To add a logging policy to the API deployment specification that applies to a particular route (overriding the global logging policy):

    1. Insert a loggingPolicies section after the route's backend section. For example:

      
      {
        "loggingPolicies": {
          "accessLog": {
            "isEnabled": true 
          },
          "executionLog": {
            "isEnabled": true,
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            },
            "loggingPolicies": {}
          }
        ]
      }
    2. (Optional) Record a summary of every request and response that goes through the gateway to the route by including the accessLog policy in the loggingPolicies section, and setting the isEnabled property to true.

      For example:

      
      {
        "loggingPolicies": {
          "accessLog": {
            "isEnabled": true 
          },
          "executionLog": {
            "isEnabled": true,
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            },
            "loggingPolicies": {
              "accessLog": {
                "isEnabled": true 
              }
            }
          }
        ]
      }
    3. (Optional) Record information about processing within the API gateway for the route by including the executionLog policy in the loggingPolicies section, setting the isEnabled property to true, and setting the logLevel property to one of the following:

      • INFO to record a summary of every processing stage.
      • WARN to record only transient errors that occur during processing. For example, a connection reset.
      • ERROR to record only persistent errors that occur during processing. For example, an internal error, or a call to a function that returns a 404 message.

      For example:

      
      {
        "loggingPolicies": {
          "accessLog": {
            "isEnabled": true 
          },
          "executionLog": {
            "isEnabled": true,
            "logLevel": "INFO"
          }
        },
        "routes": [
          {
            "path": "/hello",
            "methods": ["GET"],
            "backend": {
               "type": "ORACLE_FUNCTIONS_BACKEND",
               "functionId": "ocid1.fnfunc.oc1.phx.aaaaaaaaab______xmq"
            },
            "loggingPolicies": {
              "accessLog": {
                "isEnabled": true 
              },
              "executionLog": {
                "isEnabled": true,
                "logLevel": "ERROR"
              }
            }
          }
        ]
      }
  4. Save the JSON file containing the API deployment specification.

  5. Use the API deployment specification when you create or update an API deployment in the following ways:

    • by specifying the JSON file in the Console when you select the Upload an existing API option
    • by specifying the JSON file in a request to the API Gateway REST API

    For more information, see Deploying an API on an API Gateway by Creating an API Deployment.

Viewing Logs

Having added logging to an API deployment specification and deployed the API on an API gateway, the API Gateway service writes logs accordingly.

Viewing Logs in Oracle Cloud Infrastructure Logging

You can view the content of an API deployment log in Oracle Cloud Infrastructure Logging from the API Deployment Details page. Under Resources, click Logs, and then click the name of the log you want to view.

Alternatively, you can view the content of an API deployment log from the Oracle Cloud Infrastructure Logging Log Search page. See To view the contents of logs.

For more information about the content of access and execution logs, see:

Viewing Logs in Oracle Cloud Infrastructure Object Storage

In earlier API Gateway releases (prior to the release of the Oracle Cloud Infrastructure Logging service), you could direct API Gateway to store and view access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage. This functionality is still available for any API deployments you created in earlier releases where you previously specified a logging policy, as described below. However, note the following:
  • If you set up Oracle Cloud Infrastructure Logging to store logs for an API deployment, you are no longer able to store logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage for that API deployment.
  • The ability to store access logs and execution logs as objects in a storage bucket in Oracle Cloud Infrastructure Object Storage will be deprecated in a future release.

To view logs that have been stored in a storage bucket in Oracle Cloud Infrastructure Object Storage:

  1. In the Console, click Storage. Under Object Storage, click Buckets.
  2. Choose the Compartment that owns API Gateway-related resources. If the API gateway and the API deployment are in different compartments, choose the compartment that owns the API gateway.

    The Object Storage page shows the storage buckets in the compartment.

  3. On the Object Storage page, click the storage bucket name that includes the OCID of the compartment that owns the API Gateway-related resources.

  4. Click Objects under Resources to see a list of log objects in the storage bucket, each with a name in the format:

    api_gateway_deployment_access_log/<deployment-ocid>/<datetimestamp>.log.gz

    where:

    • <deployment-ocid> is the OCID of the API deployment.
    • <datetimestamp> is the date and time at the start of the logging window.
  5. Click the Actions icon (three dots) beside the log you want to view, and then click Download and save the log as a file in a convenient location.
  6. Open the log file in a text editor to view the log messages.

    Each log message contains:

    • the number of bytes sent in the message
    • the compartment OCID
    • the API deployment OCID
    • the API gateway OCID
    • the user agent
    • the request and the endpoint
    • the client IP address
    • the request duration (in seconds)
    • the message number
    • the timestamp

    For example:

    {"bodybytesSent":292,"compartmentId":"ocid1.compartment.oc1..aaaaaaaa7______ysq","deploymentId":"ocid1.apideployment.oc1.phx.aaaaaaaaab______pwa","gatewayId":"ocid1.apigateway.oc1.phxaaaaaaaa7______ysq","httpReferrer":"","httpUserAgent":"curl/7.54.0","msg":"GET /marketing/hello HTTP/1.1","remoteAddr":"123.45.678.90","remoteUser":"","requestDuration":"0.167","requestId":"0d5afahl6lkje4rto9tqp896004","status":200,"ts":1560954177210}