Log Partitioning is available in all regions and realms where Oracle Logging Analytics is available.

If your use case requires the partitioning feature, then file a Service Request with Oracle Support requesting this feature in Oracle Logging Analytics.

You may be collecting logs either through standalone Management Agent or using it as part of the Oracle Cloud Agent.

1. There are multiple ways for you to define the log set by editing the emd.properties file for the Management Agent. Select from one of the following:

   • You can set a static log set value for all the logs collected by the agent. Add the following property to the agent emd.properties file to specify the agent-wide log set:

     loganalytics.src.addl_src_ptn_configs=logset=<Value>

   • Set your log set value as a part of the log file name. You can specify in the agent configuration a regular expression to define how to extract part of the log file path as the log set string. In the agent emd.properties configuration file, specify the properties logsetkey and logSetExtRegex:

     loganalytics.src.addl_src_ptn_configs=logsetkey=logorigin,logSetExtRegex=<regex with a capture group>

     logorigin is the log file path.

   • You can specify to apply different log set extractions for each source separately by adding the source id as a property to the settings above.

     //sets this log set value only for source 201904301
     loganalytics.src.addl_src_ptn_configs=srcid=201904301,logset=<Value>

     //sets this log set value based on regular expression extraction only for source 201904301
     loganalytics.src.addl_src_ptn_configs=srcid=201904301,logsetkey=logorigin,logSetExtRegex=<regex with a capture group>

2. In addition to specifying the log set by using any of the methods above, include the following property in the emd.properties configuration file:

   loganalytics.src.override_config=true

3. Restart the agent after editing the configuration file.

Currently, this feature is supported only for custom logs collected from OCI Logging service into Oracle Logging Analytics using Service Connector Hub.

To set the log set value, the custom log must add a new field to the data{} block of the OCI Logging Unified Logging Format data. For example,

{
  "data": {
    "logSet": "<Value>",
    ...
  },
  "oracle": {
    ...
  },
    ...
}

When creating an Object Storage collection rule, the same options used for populating the log set in Management Agent collection are applicable.

• You can set a static log set value for all the logs collected by this object storage collection rule. Add the following property to the object storage collection rule JSON:

  {
      "name": "<Object Collection Rule Name>",
      "logSet": "<Value>",
      ...
  }

• You can specify a regular expression to parse the log set from the object path. For example:

  {
      "name": "<Object Collection Rule Name>",
      "logSetExtRegex": "\\\/o\\\/[^\\\/]*\\\/([^\\\/]*)",
      ...
  }

• If you are using overrides in the Object Collection Rule, you can also set the log set extraction for the override. In the following example, all the logs collected from the bucket will get the log set Value1 except those objects that match the override containing db. Those logs will get a log set string captured from the regular expression on the object path.

  {
      "name": "<Object Collection Rule Name>",
      "logSet": "Value1",
      "overrides": {
          "items": [{
              "matchType": "contains",
              "matchValue": "db",
              "propertyName": "logSetExtRegex",
              "propertyValue": "\\\/o\\\/[^\\\/]*\\\/([^\\\/]*)"
          }]
      }
  }

When posting the log data files to the Log Events REST API endpoint /actions/uploadLogEventsFile, you can specify the log set in the POST REST API call parameters. All the log data uploaded in this single API call will be stored with the same log set.

For example:

POST /20200601/namespaces/<namespaceName>/actions/uploadLogEventsFile?logGroupId=<logGroup_OCID>&payloadType=JSON&logSet=<Value>

When using the fluentd collector, the log set can be extracted from any other field (keyname). Specify a regular expression that defines how to capture the expected log set value from the field referenced by key_name into the field oci_la_log_set.

<filter oci.source>                  
  @type parser
  key_name oci_la_log_path      # The expression will be applied on the key "oci_la_log_path". You can pick any field that fluentd has parsed here.
  <parse>
    @type regexp
    expression '.*\/(?<oci_la_log_set>[^\.]{1,40}).*'   # Valid reg-ex for extraction
  </parse>
</filter>

When the value of the oci_la_log_path is /n/axs4r325r2ct/b/logevents/o/fileType/logSetABC.log, then the extracted log set value based on the above regex will be logSetABC.