Jump to

• Request
• Response
• Examples

Get latest run history for a data flow

get

/api/20210901/catalog/dataflows/{dataflowId}/history/latest

Return the most recent run history for a given data flow. Includes an option to return job history and task history for the latest run or return only job history. Job history includes details such as job name, job ID, status, type, start time, end time, run duration, queue duration, origin, and number of tasks. Task history includes details such as task name, status, source and target, start time, end time, run duration, size of data processed, read throughout, and number of successful rows.

Request

Path Parameters

• dataflowId(required): string(byte)
  The object ID of the data flow with Base64URL encoding. For example, if your data flow has the object ID 'User1'.'dataflow1', enter the Base64URL encoded value, J1VzZXIxJy4nZGF0YWZsb3cxJw==.

Query Parameters

• includeTasks: boolean(byte)
  Specifies whether to return historical task and job information or only the job history. Possible values are true or false. Specify true, to return both task history and job history. Specify false, to return only job history. The default value is true.

There's no request body for this operation.

Back to Top

Response

Supported Media Types

• application/json

200 Response

Successful operation.

Body ()

Root Schema : schema

Type: array

Show Source

• Array of: object jobInformation

{
    "items":{
        "type":"array",
        "$ref":"#/components/schemas/jobInformation"
    }
}

Nested Schema : jobInformation

Type: object

Show Source

• jobEndTimestamp: string
  Time job finished.
• jobExecutionTimestamp: string
  Time job started.
• jobIdentifier: string
  Identifier for the job.
• jobName: string
  Name of the job.
• jobNumberOfTasks: number
  Number of tasks in the job.
• jobOriginOfRequest: string
  Source of the job request. Either Manual or Schedule.
• jobQueuedDuration: string
  Length of time (in seconds) that the job spent in the queue.
• jobRunDuration: string
  Length of time taken (in seconds) for the job to run.
• jobRunType: string
  Type of job. For example: DATASET, DATAFLOW, SEQUENCE.
• jobStartTimestamp: string
  Time job requested.
• jobStatus: string
  Status of the job. For example: ACCEPTED, IN_PROGRESS, FAILED, SUCCEEDED, CANCELING, CANCELED.
• jobTasks: jobTasks

{
    "properties":{
        "jobName":{
            "type":"string",
            "description":"Name of the job."
        },
        "jobIdentifier":{
            "type":"string",
            "description":"Identifier for the job."
        },
        "jobRunType":{
            "type":"string",
            "description":"Type of job. For example: DATASET, DATAFLOW, SEQUENCE.\n"
        },
        "jobStatus":{
            "type":"string",
            "description":"Status of the job. For example: ACCEPTED, IN_PROGRESS, FAILED, SUCCEEDED, CANCELING, CANCELED.\n"
        },
        "jobStartTimestamp":{
            "type":"string",
            "description":"Time job requested."
        },
        "jobExecutionTimestamp":{
            "type":"string",
            "description":"Time job started."
        },
        "jobEndTimestamp":{
            "type":"string",
            "description":"Time job finished."
        },
        "jobQueuedDuration":{
            "type":"string",
            "description":"Length of time (in seconds) that the job spent in the queue."
        },
        "jobRunDuration":{
            "type":"string",
            "description":"Length of time taken (in seconds) for the job to run."
        },
        "jobOriginOfRequest":{
            "type":"string",
            "description":"Source of the job request. Either Manual or Schedule."
        },
        "jobNumberOfTasks":{
            "type":"number",
            "description":"Number of tasks in the job."
        },
        "jobTasks":{
            "items":{
                "type":"array",
                "$ref":"#/components/schemas/taskInformation"
            }
        }
    }
}

Nested Schema : jobTasks

Type: array

Show Source

• Array of: object taskInformation

{
    "items":{
        "type":"array",
        "$ref":"#/components/schemas/taskInformation"
    }
}

Nested Schema : taskInformation

Type: object

Show Source

• taskAverageReadThroughput: number
  Average read throughput for the task.
• taskDataSizeInBytes: number
  Total data processed by the task in bytes.
• taskEndTimestamp: string
  Time the task finished.
• taskName: string
  Name of the task.
• taskNumberOfSuccessfulRows: number
  Number of rows successfully processed by this task.
• taskRunDuration: string
  Length of time taken (in seconds) for the task to run.
• taskSourceName: string
  Name of the source connection.
• taskStartTimestamp: string
  Time the task started.
• taskStatus: string
  Status of the task. For example: ACCEPTED, IN_PROGRESS, FAILED, SUCCEEDED, CANCELING, CANCELED.
• taskTargetName: string
  Name of the target connection.

{
    "properties":{
        "taskName":{
            "type":"string",
            "description":"Name of the task."
        },
        "taskSourceName":{
            "type":"string",
            "description":"Name of the source connection."
        },
        "taskTargetName":{
            "type":"string",
            "description":"Name of the target connection."
        },
        "taskStatus":{
            "type":"string",
            "description":"Status of the task. For example: ACCEPTED, IN_PROGRESS, FAILED, SUCCEEDED, CANCELING, CANCELED.\n"
        },
        "taskDataSizeInBytes":{
            "type":"number",
            "description":"Total data processed by the task in bytes."
        },
        "taskStartTimestamp":{
            "type":"string",
            "description":"Time the task started."
        },
        "taskEndTimestamp":{
            "type":"string",
            "description":"Time the task finished."
        },
        "taskAverageReadThroughput":{
            "type":"number",
            "description":"Average read throughput for the task."
        },
        "taskRunDuration":{
            "type":"string",
            "description":"Length of time taken (in seconds) for the task to run."
        },
        "taskNumberOfSuccessfulRows":{
            "type":"number",
            "description":"Number of rows successfully processed by this task."
        }
    }
}

403 Response

Forbidden (missing permissions for operation, request denied for unspecified reason, and so on).

Body ()

Root Schema : Error

Type: object

Show Source

• code(required): string
  Short error code that defines the error, meant for programmatic parsing.
• message(required): string
  Human-readable error string.

{
    "type":"object",
    "properties":{
        "code":{
            "type":"string",
            "description":"Short error code that defines the error, meant for programmatic parsing.\n"
        },
        "message":{
            "type":"string",
            "description":"Human-readable error string.\n"
        }
    },
    "required":[
        "code",
        "message"
    ]
}

404 Response

Not Found. The requested resource was not found.

Body ()

Root Schema : Error

Type: object

Show Source

• code(required): string
  Short error code that defines the error, meant for programmatic parsing.
• message(required): string
  Human-readable error string.

{
    "type":"object",
    "properties":{
        "code":{
            "type":"string",
            "description":"Short error code that defines the error, meant for programmatic parsing.\n"
        },
        "message":{
            "type":"string",
            "description":"Human-readable error string.\n"
        }
    },
    "required":[
        "code",
        "message"
    ]
}

500 Response

Internal Server Error. The server encountered an unexpected condition preventing fulfilment of the request.

Body ()

Root Schema : Error

Type: object

Show Source

• code(required): string
  Short error code that defines the error, meant for programmatic parsing.
• message(required): string
  Human-readable error string.

{
    "type":"object",
    "properties":{
        "code":{
            "type":"string",
            "description":"Short error code that defines the error, meant for programmatic parsing.\n"
        },
        "message":{
            "type":"string",
            "description":"Human-readable error string.\n"
        }
    },
    "required":[
        "code",
        "message"
    ]
}

Back to Top

Examples

In this example, you return the detailed history of the most recent job run for a given dataflowID. The object ID of the data flow is '[e-mail]'.'MySalesDataFlow: Enrich MySales Report' which has the Base64URL-safe encoded dataflowID value: J2pvaG4uc21pdGhAbXljb21wYW55LmNvbScuJ015U2FsZXNEYXRhRmxvdzogRW5yaWNoIE15U2FsZXMgUmVwb3J0Jw.

cURL Example:

curl -i \
     --header 'Authorization: Bearer <token>' \
     --header 'Content-Type: application/json' \
     --request GET 'https://<hostname>/api/20210901/catalog/dataflows/J2pvaG4uc21pdGhAbXljb21wYW55LmNvbScuJ015U2FsZXNEYXRhRmxvdzogRW5yaWNoIE15U2FsZXMgUmVwb3J0Jw/history/latest' \

This example returns both job and task history for the latest run.

Example of Request Body

Not applicable.

Example of Response Body

[
    {
    "jobName":"'[e-mail]'.'MySalesDataFlow: Enrich MySales Report'",
    "jobIdentifier":"1234ABCD5678EFGH9012IJKL3456MNOP",
    "jobStartTimestamp":"2024-05-14T18:30:50.0+0000",
    "jobExecutionStartTimestamp":"2024-05-14T18:31:07.0+0000",
    "jobEndTimestamp":"2024-05-14T18:31:25.0+0000",
    "jobStatus":"SUCCEEDED",
    "jobRunType":"DATAFLOW",
    "jobQueuedDuration":"PT35S",
    "jobRunDuration":"PT18S",
    "jobOriginOfRequest":"Manual",
    "jobNumberOfTasks":"1",
    "jobTasks":
        [{"taskName":"DFLW : MySalesDataFlow: Enrich MySales Report",
        "taskStartTimestamp":"2024-05-14T18:31:10.0+0000",
        "taskEndTimestamp":"2024-05-14T18:31:25.0+0000",
        "taskStatus":"SUCCEEDED",
        "taskDuration":"PT15S"}]
    }
]

cURL Example with Parameters:

If you want to exclude task history, apply the query parameter includeTasks=false.

curl -i \
     --header 'Authorization: Bearer <token>' \
     --header 'Content-Type: application/json' \
     --request GET 'https://<hostname>/api/20210901/catalog/dataflows/J2pvaG4uc21pdGhAbXljb21wYW55LmNvbScuJ015U2FsZXNEYXRhRmxvdzogRW5yaWNoIE15U2FsZXMgUmVwb3J0Jw/history/latest?includeTasks=false'

Back to Top