Creating Segment Export Jobs

Segment Export job type provides a way to execute mcpsquery and export resulting dataset to a configured sink. Primary use-case is to export segments to a marketing automation tool.

  1. User builds mcpsQuery and persists MCPSQuery object in metadata
  2. User creates Job metadata with JobType = QUERYEXPORT, uniqueId of MCPSQuery, sinkinstanceId, ttlInSeconds. ttlInSeconds is optional and it determines data retention period for the exported data in the MCPS system (not the external system). By default, ttlnSeconds is set to 24 hrs (86400 seconds). Optionally, user can add reference to HTTP notification in the job metadata. Optionally, user can set schedule in job metadata.
  3. User publishes the tenant.

  4. Job execution: MCPSQuery (identified as mcpsQueryId in Job metadata) is run and the results of the query are exported to the configured sink.

    • NOTE: Currently, we support FTP and SFTP sink with CSV, JSON formats. Compression supported is gzip. We are working on Responsys sink to be available shortly.
    • Once data export to the sink is complete, the configured notifications are sent out.

Refer to the Job execution APIs to execute the job.

Tasks

Create the MCPSQuery.

User Action Method API endpoint Request Payload
Create MCPSQuery object POST POST metdata/mcpsqueries 
{
 "MCPSQuery":{
            "tenantId": 302,
            "name": "selectCustomer",
            "versionTS": 1551210321895,
            "active": true,
            "createdTS": 1551210321895,
            "uniqueId":"selectCustomer",
            "operation":{
                ...
 
            ...
    }
}
Update MCPSQuery object PUT POST metadata/mcpsqueries/<object> 
{
 "MCPSQuery":{
            "tenantId": 302,
            "name": "selectCustomer",
            "versionTS": 1551210321895,
            "active": true,
            "createdTS": 1551210321895,
            "uniqueId":"selectCustomer",
            "operation":{
                ...
 
            ...
    }
}
Get MCPSQuery metadata GET GET metadata/mcpsqueries/<object>

N/A

Create Job metadata with JobType = QUERYEXPORT.

User Action Method API endpoint Request Payload
Create Job metadata POST POST /metadata/jobs

 

{
    "tenantId": 302,
    "name": "Customer1_new",
    "versionTS": 1553555128414,
    "description": "Segment export job to test different customer attributes",
    "active": true,
    "createdTS": 1553555128414,
    "jobId": "Customer1_new",
    "jobData": {
        "ctype": ".QueryExportJobData",
        "templateInfo": {
            "workflowComponents": [
            {
                "left": "SINK",
                "right": "FTPSink_csv"
            }
            ]
        },
        "mcpsQueryId": "selectCustomer",
        "ttlInSeconds": 36000000
    },
    "notifications": [
    {
        "left": "HTTP",
        "right": "Notification1"
    }
 ]
 }
Update QueryExport Job metadata PUT PUT metadata/jobs/QUERYEXPORT 
{
    "tenantId": 302,
    "name": "Customer1_new",
    "versionTS": 1553555128414,
    "description": "Segment export job to test different customer attributes",
    "active": true,
    "createdTS": 1553555128414,
    "jobId": "Customer1_new",
    "jobData": {
        "ctype": ".QueryExportJobData",
        "templateInfo": {
        "workflowComponents": [
        {
            "left": "SINK",
            "right": "FTPSink_csv"
        }
        ]
    },
    "mcpsQueryId": "selectCustomer",
    "ttlInSeconds": 36000000
    },
    "notifications": [
    {
    "left": "HTTP",
    "right": "Notification1"
    }
    ]
 }
Get Job metadata GET GET /metadata/jobs/{jobType}/{jobId}

N/A

Learn more

Jobs API