Manage Trail files

OCI GoldenGate Trail files quickly add up over time. Without purge tasks in place to manage these trail files, daily backups will take exponentially longer to complete and use compute resources that could otherwise be used elsewhere.

Note:

This information applies only to Data replication deployments.

Related Topics

View Trail files

You can view Trail files details such as size, sequence, producers, and consumers on the deployment details page.

To view Trail files:
  1. From the GoldenGate Overview page, click Deployments.
  2. On the Deployments page, select a deployment to view its details, or select View details from the deployment's Action menu (ellipsis icon).
  3. On the Deployment details page, under Resources, click View Trail files.

Use this information to identify Trail files that are no longer used or needed, and then create Purge tasks to manage them.

Encrypt Trail files

Use master encryption keys to encrypt trail files distributed to other GoldenGate deployments. See Manage Master encryption key wallets to learn more.

Purge Trail files

OCI GoldenGate Trail files quickly add up over time. Without purge tasks in place to manage these Trail files, daily backups will take exponentially longer to complete and use compute resources that could otherwise be used elsewhere.

Oracle recommends that you create a backup first, before purging Trail files. See Create a manual backup. You can then download the backup and review its contents (<deployment-name>/var/lib/data/).
To purge OCI GoldenGate Trail files:
  1. In the OCI GoldenGate deployment console, review the following processes that generate Trail files, and take note of the Trail files you want to keep:
    • In the Administration Service, review the details of each Extract and Replicat. (Click the process name, and then select Details.)
    • In the Distribution Service, review the details of each Distribution Path (if any).
    • In the Receiver Service, review the details of each Receiver Path (if any).

    All other Trail files not on your list can be purged.

  2. Set up Purge Tasks to clean up unused Trail files in the deployment console.
    1. In the OCI GoldenGate deployment console, open the navigation menu (hamburger icon) for the Administration Service, and then select Configuration.
    2. On the Configuration page, click Tasks, and then click Purge Trails.
    3. Click Add Purge Trails Task (plus icon).
    4. Under Create a new Purge Trails task, complete the following fields, and then click Submit:
      1. For Operation Name, enter a name for the purge task.
      2. For Trail, enter a name of a Trail file, and then click Add Trail (plus icon). Repeat this step to add more Trail files. Selected Trails populates with the names of the Trail files as you add them.
      3. Keep Use Checkpoints enabled if you want to purge after all Extract and Replicat processes are finished with the file(s), as indicated by checkpoints. Disabling this option allows purging without considering checkpoints and a minimum of one file (if no MIN value is specified) or the number of files specifed with MIN are kept.

        Note:

        Orphan Checkpoint files can't be deleted in the deployment console. Use REST APIs to delete orphan Checkpoint files.
      4. For Keep Rule, specify the Hours, Days, or Number of Files to keep.
      5. For Purge Frequency, specify the frequency to run this purge task.

    You can add more Purge Tasks or disable them as needed.

  3. Clean up unused Trail files using Admin Client.
    1. On the deployment details page, click Launch Admin Client.
      If connecting to an OCI GoldenGate deployment with a private endpoint, follow the instructions in Connect to Admin Client through Cloud Shell.
    2. Run the following command to purge trail files no longer used by Extracts:
      purge exttrail <trail-file-name>
      For more information, see PURGE EXTTRAIL in the GoldenGate Command Line Reference guide.
  4. Clean up unused Trail files using REST APIs.
    1. Open Cloud Shell.
    2. In Cloud Shell, run the following REST API call to the OCI GoldenGate deployment. Ensure that you replace the placeholders with your deployment's actual values.
      curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type:
      application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ {
      "name": "<trail_file_name>" } ], "useCheckpoints": false, "keep": [ { "type":
      "min", "units": "files", "value": 0 } ]}'

To ensure all Trail files are deleted, you can do one of the following:

  • On the deployment details page in the Oracle Cloud console, under Resources, click Trail files and then click Refresh to review the list of Trail files.
  • Create another manual backup and check the contents of the backup for the Trail files you wanted to keep. Note the aggregate size reduction and speed that the backup now completes.

REST Examples

The following examples show how to remove Trail files under different conditions:

Example 1: Purge all Trail files whose names match "IL":

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "IL" } ], "useCheckpoints": false, "keep": [ { "type": "min", "units": "files", "value": 0 } ]}'

Example 2: Purge all Trail files, as indicated by the asterisk (*), within the subdirectory, demo: 

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "*", "path": "/u02/Deployment/var/lib/data/demo" } ], "useCheckpoints": false, "keep": [ { "type": "min", "units": "files", "value": 0 } ]}'

Example 3: Purge all L1 Extract Trail files:

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "L1" } ], "useCheckpoints": false, "keep": [ { "type": "min", "units": "files", "value": 0 } ]}'

Example 4: Purge all Trail files whose names match "L1" but keep files from the last 10 hours:

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "L1" } ], "useCheckpoints": false, "keep": [ { "type": "min", "units": "hours", "value": 10 } ]}'

Example 5: Purge all L1 Extract Trail files more than a day old using USECHECKPOINTS:

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "L1" } ], "useCheckpoints": true, "keep": [ { "type": "min", "units": "days", "value": 1 } ]}'

Example 6: Purge all L1 Extract Trail files that are more than a day old: 

curl -X POST https://<domain-name>/services/v2/commands/execute -n -H 'Content-Type: application/json' -d '{ "name": "purge", "purgeType": "trails", "trails": [ { "name": "L1" } ], "useCheckpoints": false, "keep": [ { "type": "min", "units": "hours", "value": 23 } ]}'