Custom Data Source Analyzer API

The Custom Data Source Analyzer API (CDSA API) is a programming interface that enables you, our customer, to pass your own data into Listen. Your data is kept securely and is not visible to other customers.

By adding your data into Listen, it will be fully analyzed against Topics and Indicators, with the results visible in the Listen dashboard.

In this article:

About Custom Data Source Analyzer API

Using the Custom Data Source Analyzer
    · Before Using the CDSA API
    · Preparing Your Data
    · Checking Your Data
    · Loading Your Data
    · Viewing Your Data
    · Routing to Engage

About General Guidelines
    · URL
    · Encryption
    · Authentication
    · Error Messages

About the CDSA API Data Schema

About the Types of CDSA API Call
    · Production Workflow API Calls
    · Test Workflow API Calls
    · Metadata API Calls

About API Calls
    · Job Creation
    · Job Status
    · Data Validation
    · Version URI Metadata
    · Job Creation Metadata
    · Job Status Metadata
    · Validate Data Metadata
    · Version Sub-resources

Sample Payload File

Click a link above to get directly to that section, or scroll down and start reading.

About Custom Data Source Analyzer API

The Custom Data Source Analyzer API (CDSA API) allows you to load your own private data into the Listen component of SRM, where it is analyzed and displayed in the Dashboard alongside public social media data. The CDSA API uses a set of API calls using HTTP BASIC authentication over SSL, which processes data provided in a JSON format.

What can this API do?

The CDSA API allows you to view analysis of your own data in Listen alongside the social media posts that Listen provides as standard.

The CDSA API can accept data from any source that has unstructured text that can be analyzed, together with an author.

Data loaded through the CDSA API undergoes the same processing and analysis as the standard social media data in Listen, and is presented in the same screens. The only differences are that data loaded through the CDSA API is only ever visible to your users, will never be shared in any way with other companies using Oracle SEM, and that CDSA API data is displayed as a custom data type in the Listen & Analyze dashboard.

Audience

This developer's guide is intended for software developers that need a technical guide for interacting with the Custom Data Source Analyzer API.

Using the Custom Data Source Analyzer API

Before Using the CDSA API

Before using the CDSA API, make sure you have the necessary pre-requisites:

SE&M & CDSA API Licenses

The CDSA API is a feature of Oracle Social Engagement and Monitoring (SE&M). The CDSA API cannot be used without a valid SE&M license, and access to the SE&M dashboard is necessary in order to view the data loaded by SE&M.
The CDSA API is a separately licensed component of SE&M. An additional CDSA API license must be purchased in order to load data through the CDSA API.

License Keys

Use of the CDSA API requires two license keys:

Customer Key
API Key

These are provided to licensed CDSA API users and are accessed through the Listen application. You can find them on the All Topics page of Listen & Analyze.

To access the CDSA license keys:

Navigate to Listen & Analyze.
Click the Topics tab to view all topics.
Click the Get API Key link on the right side of the page.

Custom Topics

When you load your data through the CDSA API, it is displayed in a Listen & Analyze Dashboard for a specific Listen topic. Before you add your data, you will need to create a Listen topic specifically for your data. (Complete instructions on creating a Listen topic can be found here.)

When you create a topic for CDSA API data, you have to configure certain settings to make sure your data will display correctly. Under Advanced Options, in the Topic Scope section, click Custom.

Note: Make sure that when you create your topic you match all the criteria that your CDSA API data has, including in particular that the topic has search terms to match all your CDSA API messages. Your CDSA API data will not display correctly unless it matches the topic criteria.

Preparing Your Data

The CDSA API can process a rich set of data, however it must be presented in the format expected by the CDSA API. This means you will need to match your data schema to the CDSA API schema (see the CDSA API schema here).

The data will need to be presented in a JSON format. JSON is a widely used standard for data interchange. Data is loaded through the CDSA API in Jobs, with each Job consisting of a JSON document containing one or more messages.

The maximum number of messages in a batch is 15000 messages.

Note: If you are having difficulty getting your data into a valid JSON format the JSONLint website can often help.

There are a few things that are important to consider when loading data into the CDSA API:

Language Identification:
Messages will only be loaded if they are identified as using a supported language. If you know the language of a message, then the best approach is to specify this using the ‘languageTag’ attribute. This will ensure that the message is correctly identified with the required language and loaded accordingly.
If you don’t know the language of each message, for example if you have a lot of messages using many different languages, then the CDSA API will automatically identify the language for you. Automatic language identification is performed for all messages that do not have the ‘languageTag’ specified. Automatic language identification is highly accurate, but cannot be guaranteed to be 100% accurate, especially with very short messages with few words, or messages that use a lot of jargon or abbreviations. If no ‘languageTag’ is specified, and the language cannot be automatically identified (or is identified as an unsupported language), then the message will not be loaded.
Character Set:
The CDSA API can only process data that uses the UTF-8 character set. UTF-8 is a widely used standard that allows the use of characters from all languages around the world, however sometimes data can include characters that are not part of UTF-8, especially when that data comes from unknown sources or many different sources. If data uploaded to the CDSA API includes non-UTF-8 characters then the whole job is likely to fail.
To avoid any problems make sure that you use ‘UTF-8’ as the character set when you’re saving your data for upload. You can clean out non-UTF-8 characters from files using the following command on Linux:
> iconv -f utf-8 -t utf-8 –c
Required Fields:
Messages will only be loaded if they have all the required fields. These are:
- At least one Text field (can be either ‘Subject’ or ‘Body’)
- Source URL
- Date/Time created (publishedAt)
Some fields require data to be provided in a particular format; for example, the "languageTag" must be a two-letter code for a supported language (i.e. “en”). The "Constraints" column of the Schema section of this document gives details of any specific formats required by the CDSA API.

Checking Your Data

Before you load your data, we strongly recommend you validate the data first. This is especially important when you are developing a new integration to the CDSA API. Validation uses a special API call that accepts and checks CDSA API data, but does not actually load anything.

Validation offers the following main benefits:

In the case of any problem with the data it provides details of what is causing the problem.
Once data is loaded through the CDSA API it can’t be removed. Validation allows testing to be performed without affecting live data.

Loading the Data

Once your data has been validated, you can load into Listen & Analyze through the CDSA API. Make sure you have your active topic with Custom Scope created before performing any data load. If there are no schema violations in the data, the data will be loaded successfully. If there are any errors in the data, then the error details are returned by the CDS API. The user will then have to fix the errors and reload the data.

Note: The upload size should not exceed 25 MB.

Viewing the Data

Once your data has been successfully loaded, it can be viewed in Listen & Analyze dashboards. Once you are in the Dashboards section, click the topic you created for the data, and your data will appear. If you use the date range selector, the data will appear for the date specified in the publishedAt field when the data was loaded.

Routing CDSA API Messages to Engage

Note: If you want to route CDSA API messages to Engage:

All messages must be within the last 7 days.
All messages must have a message URL.

About General Guidelines

URL

All CDSA API calls use the same host (api.collectiveintellect.com) and all use HTTPS exclusively. All URLs follow the same format:

https://api.collectiveintellect.com/customDataSourceApi

Encryption

All CDSA API calls are encrypted using HTTP/SSL. This ensures that both the header (including the license keys) and the payload are secured at all times.

Authentication

All CDSA API calls require HTTP BASIC Authentication using the following credentials:

User: Customer Key
Password: API Key

Error Messages

The CDSA API supports the standard HTTP response codes:

200 OK: Request successfully executed and the response has content.
201 Created: Resource successfully created. The response contains the created resource.
204 No Content: Request successfully executed and the response doesn't have content.
304 Not Modified: Document has not been modified for a conditional GET request.
400 Bad Request: The request could not be understood by the server due to malformed syntax.
404 Not Found: The requested resource was not found.
413 Request Entity Too Large: The uploaded data is too large to process
422 Unprocessable Entity: There is an error in the data being uploaded
500 Internal Server Error: The server encountered an unexpected condition which prevented it from fulfilling the request.

In addition we provide more detailed information when the validate API call is used. This is covered in the API calls section of this document.

Sample Errors Found Response

Response Entity:


{          

  “type”: “http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1”,

  “title”: “Json validation failure.”,

  “o:errorDetails”: [

    {

      “type:  “http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1”,

      “title”: “object has missing required properties ([\”publishedAt\”])”,

      “o:errorPath”: “\”/entry/1\””,

      “messageId”: 1

    }

  ]

}

Response Headers:
HTTP/1.1 422 Unprocessable Entity
Content-Language: en
Content-Type: application/json

About the CDSA API Data Schema

All data loaded into the CDSA API must be in JSON format and must conform to a fixed schema, which is described below:

Category	Name	Required	Type	Function	Constraints
Message	passThroughId		String	Customer defined ID for the message.
	publishedAt	Y	Date/Time	Date/time message was created. Will be used for the message date/time in the dashboard.	ISO 8601 compliant See http://tools.ietf.org/html/rfc3339#section-5.6 Valid dates: 1990-12-31T23:59:59Z 1990-12-31T23:59:59.1Z 1990-12-31T23:59:59.12Z 1990-12-31T23:59:59.123Z 1990-12-31T23:59:59-08:00 1990-12-31T23:59:59.1-08:00 1990-12-31T23:59:59.12-08:00 1990-12-31T23:59:59.123-08:00 Invalid Dates: 1990-12-31T23:59:59.1234Z 1990-12-31T23:59:59.1234-08:00 Should not be before unix epoch, that is: Jan 01, 1970, 00:00 UTC ( 1970-01-01T00:00:00.000Z ) Date-time should be past values, future date-time values are invalid
	url		URI	URL of message	maxLength:10000
	parentId		Integer	ID of parent message.	Range: 0 to 1099511627775 (40 bit)
	languageTag		String	Native language of message. The language given here will always be used. If no language is given then the language will be automatically identified.	Valid values are a custom subset of IETF language tags as described in RFC 4646 (IANA codes).
	location		Location	The location of the message, with both latitude and longitude.	Type 'Location' described below
	author		Author	The author of the message.	Type 'Author' described below
	source	Y	Type 'Source' described below	The source of the message.	message.source
	texts	Y	Type 'CommonNamedText' described below Minimum : 1	The text used in the message.	message.named_texts maxLength of all text fields is 30000
Location	displayName		String	Location name	Fluid field identifying the location
	country		String	Country code	ISO codes country is mandatory only when region is given, otherwise it is optional
	region		String	Region
	subRegion		String	Sub-region
	locality		String	Locality
	longitude		Float	Longitude coordinate	-180 to 180 Note: Either both longitude & latitude should be given or neither should
	latitude		Float	Latitude coordinate	-90 to 90 Note: Either both longitude & latitude should be given or neither should
Author	name		String	Author name
	displayName		String	Username of author such as Twitter handle
	description		String	Description of author
	profileUrl		URI	Link to author profile	maxLength:10000
	email		String	Author's email address
	gender		String	Gender of author	Must be either "male" or "female" if used
	birthDate		Date/Time	Birth date of author	ISO 8601 standard Should be after 1900-01-01T00:00:00.000Z Date-time should be past values, future date-time values are invalid
	passThroughId		String	External ID of author
	createdAt		Date/Time	When the author profile was created	ISO 8601 standard Should be after 1970-01-01T00:00:000Z Date-time should be past values, future date-time values are invalid
	imageURL		URL	Link to image of author	maxLength:10000
	utcOffset		String	Author's time zone	Must be compliant with ISO 8601 time zones Recommends to use ±[hh]:[mm]. This means UTC ± the specified time.
	friendsCount		Integer	Number of friends of author	Minimum: 0
	followersCount		Integer	Number of followers of author	Minimum: 0
	sourceMessagesCount		Integer	Number of messages posted by author	Minimum: 1
	kloutScore		Integer	Author's Klout score	Minimum: 0 Maximum: 100
	location		Location	Author's location, with both latitude and longitude.
CommonNamedText	name		String	Name of text type	Must be either "body" or "subject"
	text	Y	String	Text contents	The length of all text fields summed up together should not be more than 30000 characters.
Source	name		String	Name of message source	maxLength : 100
	URL	Y	URL	Link to home page of message source	maxLength:10000

About the Types of CDSA API Call

There are two flows for running the CDSA API: Production and Test. We highly recommend you use the Test flow first before using the Production flow.

Production Workflow API Calls

These API calls are provided for production use of the CDSA API:

Create Job
Query Job Status

Test Workflow API Calls

This API call is used to test that data can be submitted to the production CDSA API without actually loading any data into the system.

Validate Job

Metadata Calls

In addition to the test & production workflow calls and in line with REST standards, a comprehensive set of metadata API calls are provided. These enable users to programmatically query the metadata associated with the CDSA API. The metadata is important if you want to make advanced uses of the CDSA API, such as building a dynamic UI on top of the CDSA API REST services, or use design time binding of UI components to REST services. The following metadata services are provided:

Version Metadata Catalog
Metadata for Job Creation
Metadata for Job Status
Metadata for Validate Data
Version Sub-Resources

About API Calls

Job Creation

Submits a CDSA API job to be loaded into Listen.
Requires a json document containing a set of messages. The message format should follow the pre-specified JSON schema.
Data can be a plain JSON media or gzipped JSON file.
The "Content-Type" header should be set according to the content type in use:
- application/json (plain JSON file - default)
- application/gzip (gzipped JSON file)
If the uploaded data is valid, then a job will be created and a job ID will be returned. You can use this job ID to track the status of the job. If there are errors in the uploaded data, then a detailed error message describing the error will be returned as the response and the job will not be created.
In cases where job creation was successful, the following is returned: id, status, and links. Links contains a metadata link which describes the resource and two more links which point to the resource.
Sample usage using Curl:
curl -u abcdef:7180fadf552a46fe –X
POST https://api.collectiveintellect.com/customDataSourceApi/v1/job -T data.json

Success Response

Response entity:

 
{
  "id": "54e27518407845e694c264c18b8bf26e",
  "status": "DataReceived",
  "links": [
    {
      "rel": "self",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
    },
    {
      "rel": "describedBy",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job"
    }
  ]
  "receivedMessagesCount": 3,
  "validMessagesCount": null,
  "categorizedMessagesCount": null,
  "errorMessage": null
}

Response Headers:
HTTP/1.1 201 Created
Content-Language: en
Content-Type: application/json
Location: https://api.collectiveintellect.com/customDataSourceApi/v1/job/a5300e4e6c6a48509f352fd5fa9dd63f

Failure Response

In cases where there are errors in the data.

Response entity:

 
{
  "status": "Failed",
  "errorMessage": "Validation failure. One or more messages does not match the prescribed schema",
  "o:errorDetails": [
    {
      "type": "https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1",
      "title": "object has missing required properties ([\"publishedAt\"])",
      "o:errorPath": "\"/1\"",
      "messageId": 1
    }
  ],
  "links": [
    {
      "rel": "self",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
    },
    {
      "rel": "describedBy",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job"
    }
  ]
}

Response Headers:
HTTP/1.1 422 Unprocessable Entity
Content-Language: en
Content-Type: application/json

Job Status

Queries the status of a previously submitted job, and returns the results for jobs that have completed.
Requires the job_id to be specified in the URL. This job_id is the ID return by the Job Creation API.
The current status of the job is returned. Valid statuses are:
- DataReceived - Job data received and queued for initial processing
- TransformProcessing - Job data undergoing initial transformation processing
- TransformCompleted - Job data has completed initial transformation processing and is queued for pipeline processing
- PipelineProcessing - Job data undergoing pipeline processing
- Completed - Job completed. Valid data available in SE&M UI
- Failed - Job failed. No data loaded.

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET
https://api.collectiveintellect.com/customDataSourceApi/v1/job/a5300e4e6c6a48509f352fd5fa9dd63f -v

Standard Response:

Response Entity:

 
{
  "id": "a5300e4e6c6a48509f352fd5fa9dd63f",
  "status": "Completed",
  "links": [
    {
      "rel": "self",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job/a5300e4e6c6a48509f352fd5fa9dd63f"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job/a5300e4e6c6a48509f352fd5fa9dd63f"
    },
    {
      "rel": "describedBy",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{jobId}"
    }
  ]
  "receivedMessagesCount": 3,
  "validMessagesCount": 3,
  "categorizedMessagesCount": 0,
  "errorMessage": null
}

Response Headers:
HTTP/1.1 200 Created
Content-Language: en
Content-Type: application/json

Data Validation

Used to check that the data to be loaded has the correct format prior to uploading.
It is strongly recommended the data validation call is used before the production call is used as once data has been loaded it cannot be removed.
Requires a json document containing a set of messages. The message format should follow the pre-specified JSON schema.
It will not create a job. It will validate the data and return the exceptions, if any.
Data can be a plain json file or gzipped json file.
The "Content-Type" header should be set according to the content type in use:
- application/json (plain JSON file - default)
- application/gzip (gzipped JSON file

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X POST
https://api.collectiveintellect.com/customDataSourceApi/v1/validate -T valid.json -v

Successful Validation Response

Response Entity:
Null

Response Header:
HTTP/1.1 204 No Content

Errors Found Response

Response Entity:


{
  “type”: “http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1”,
  “title”: “Json validation failure.”,
  “o:errorDetails”: [
    {
      “type:  “http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1”,
      “title”: “object has missing required properties ([\”publishedAt\”])”,
      “o:errorPath”: “\”/entry/1\””,
      “messageId”: 1
    }
  ]
}

Response Headers:
HTTP/1.1 422 Unprocessable Entity
Content-Language: en
Content-Type: application/json

Version URI Metadata

Metadata call returning the various versions available under the root resource.
It gives a list of all the versions available.
Response entity follows the schema specified in the Oracle's REST standard document.

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog -v

Standard Response:

Response Entity:

 
{
  "items": [
    {
      "name": "job",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job"
        }
      ]
    },
    {
      "name": "job/{jobId}",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{jobId}"
        }
      ]
    },
    {
      "name": "validate",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate"
        }
      ]
    }
  ],
  "links": [
    {
      "rel": "self",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog"
    }
  ]
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json

Job Creation Metadata

Used to get the metadata of the job creation call.

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job -v

Standard Response:

Response Entity:


{
  "name": "job",
  "links": [
    {
      "rel": "alternate",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job",
      "mediaType": "application/schema+json"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job",
      "mediaType": "application/json"
    },
    {
      "rel": "describes",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
    },
    {
      "rel": "create",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job",
      "method": "POST"
    }
  ]
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job -v -H
"Accept:application/schema+json"

Response Entity:
Json schema for the input data format.

Job Status Metadata

Used to get the metadata of the job status call.
Can optionally return the schema used for the validation by including a “Accept:application/schema+json” header

Without schema:

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{job_id} –v

Standard Response:

Response Entity:

  
{
  "name": "job/{jobId}",
  "links": [
    {
      "rel": "alternate",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{jobId}",
      "mediaType": "application/schema+json"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{jobId}",
      "mediaType": "application/json"
    },
    {
      "rel": "describes",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job/{jobId}"
    },
    {
      "rel": "query",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job/{jobId}",
      "method": "GET"
    }
  ]
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json

curl -u RGhhcm1hIFJhbyBTb25QQQ==:da945b768bdea3b0 -X GET http://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{job_id} -v -H "Accept:application/schema+json"

Including schema:

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET
https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/job/{job_id} -v -H "Accept:application/schema+json"

Response Entity:
Json schema for the response entity.

 
{
  "$schema": "http://json-schema.org/draft-04/hyper-schema#",
  "required": [
    "id",
    "status",
    "links"
  ],
  "allOf": [
    {
      "$ref": "#/definitions/singular"
    }
  ],
  "properties": {
    "id": {
      "description": "Identifier of the uploaded job.",
      "type": "string"
    },
    "status": {
      "description": "Status of the job.",
      "enum": [
        "DataReceived",
        "Failed",
        "TransformProcessing",
        "TransformComplete"
        "PipelineProcessing",
        "Completed"
      ]
    },
    "receivedMessagesCount": {
      "description": "count of messages received in the batch",
      "type": [
        "integer",
        "null"
      ]
    },
    "validMessagesCount": {
      "description": "count of messages meeting minimum requirement(s).",
      "type": [
        "integer",
        "null"
      ]
    },
    "categorizedMessagesCount": {
      "description": "count of messages successfully categorized to any of the active topics",
      "type": [
        "integer",
        "null"
      ]
    },
    "errorMessage": {
      "type": [
        "string",
        "null"
      ],
      "description": "A brief description of the error encountered in the processing
of the batch. Relevant in case of Failed job"
    }
  },
  "links": [
    {
      "href": "./",
      "rel": "self"
    },
    {
      "href": "./",
      "rel": "canonical"
    },
    {
      "href": "http://json-schema.org/draft-04/hyper-schema#",
      "rel": "describedBy"
    }
  ],
  "definitions": {
    "link": {
      "$schema": "http://json-schema.org/draft-04/hyper-schema#",
      "type": "object",
      "required": [
        "href",
        "rel"
      ],
      "description": "Metadata describing link description objects that MAY appear on
json instance representation.",
      "properties": {
        "href": {
          "description": "Either a URI [RFC3986] or a URI Template [RFC6570]. If the
value is a URI Template then the \"templated\" property should be true.",
          "type": "string"
        },
        "rel": {
          "description": "The name of the link relation that consumer may lookup as a
secondary key (in addition to the type property) to retrieve actual link details, e.g.
href or profile",
          "type": "string"
        },
        "templated": {
          "description": "Indicates the \"href\" property is a URI Template. If not
specified, assume the \"href\" is a URI.",
          "type": "boolean"
        },
        "mediaType": {
          "description": "media type (as defined by RFC 2046) describing the link
target",
          "type": "string"
        },
        "method": {
          "description": "method for requesting the target of the link (e.g. for HTTP
this might be \"GET\" or \"DELETE\")",
          "type": "string"
        },
        "profile": {
          "description": "Indiciates the json-schema describing the resource expected
when derefencing the target resource",
          "type": "string"
        }
      }
    },
    "singular": {
      "$schema": "http://json-schema.org/draft-04/hyper-schema#",
      "title": "Singular",
      "description": "Oracle base singular resource schema definition",
      "type": "object",
      "properties": {
        "links": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/link"
          }
        }
      },
      "links": [
        {
          "href": "./",
          "rel": "self"
        },
        {
          "href": "./",
          "rel": "canonical"
        },
        {
          "href": "http://json-schema.org/draft-04/hyper-schema#",
          "rel": "describedBy"
        }
      ]
    }
  }
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json; charset=UTF-8

Validate Data Metadata

Used to get the metadata of the validate data call.
Can optionally return the schema used for the validation by including a “Accept:application/schema+json” header

Without schema

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate -v

Response Entity:

  
{
  "name": "validate",
  "links": [
    {
      "rel": "alternate",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate",
      "mediaType": "application/schema+json"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate",
      "mediaType": "application/json"
    },
    {
      "rel": "describes",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/validate"
    },
    {
      "rel": "create",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/validate",
      "method": "POST"
    }
  ]
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json

curl -u RGhhcm1hIFJhbyBTb25QQQ==:da945b768bdea3b0 -X GET http://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate -v -H "Accept:application/schema+json" -v

Including schema:

Sample usage using Curl:
curl -u abcdef:7180fadf552a46fe -X
GET https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog/validate -v -H
"Accept:application/schema+json" -v
Response Entity:
Json schema for the input data format. The message format should follow the pre-specified JSON schema.

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: text/plain; charset=UTF-8 (It should be application/json)

Version Sub-Resources

Used to get the resources for a given version of the API.

Sample usage using Curl:

curl -u abcdef:7180fadf552a46fe -X GET https://api.collectiveintellect.com/customDataSourceApi/v1 -v

Response Entity:


{
  "items": [
    {
      "name": "job",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job"
        }
      ]
    },
    {
      "name": "job/{jobId}",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/job/{jobId}"
        }
      ]
    },
    {
      "name": "validate",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/validate"
        }
      ]
    },
    {
      "name": "metadata-catalog",
      "links": [
        {
          "rel": "canonical",
          "href": "https://api.collectiveintellect.com/customDataSourceApi/v1/metadata-catalog"
        }
      ]
    }
  ],
  "links": [
    {
      "rel": "self",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1"
    },
    {
      "rel": "canonical",
      "href": "https://api.collectiveintellect.com/customDataSourceApi/v1"
    }
  ]
}

Response Headers:
HTTP/1.1 200 OK
Content-Language: en
Content-Type: application/json

Sample Payload File


[
  {
    "passThroughId": "Default Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "I am looking for enterprise data only by Preethi as of June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 67890,
    "languageTag": "en"
  },
  {
    "passThroughId": "Random Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Black Friday Shopping was good said Preethi from california and test for Author Field as of June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-07-08T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 847256596408,
    "languageTag": "en"
  },
  {
    "passThroughId": "MIN Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "I hate shopping on black fridays says Preethi and test for Author Field on June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-05-07T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 0,
    "languageTag": "en"
  },
  {
    "passThroughId": "MAX Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "I dont like the weather today told Preethi and test for Author Field as of June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-05-07T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 2001,
    "languageTag": "en"
  },
  {
    "passThroughId": "Negative Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "It is raining in california on June 8th good weather to stay home, bad weather to be at work and driving on the roads said preethi. did you go black friday shopping? and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-05-07T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 1,
    "languageTag": "en"
  },
  {
    "passThroughId": "Too low Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "The weather in east coast is pretty bad when Preethi went and Preethi had a bad time and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-05-07T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 670,
    "languageTag": "en"
  },
  {
    "passThroughId": "Too Big Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-03-14T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "I found some good deals during black friday shopping and test for Author Field for Preethi on June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 679,
    "languageTag": "en"
  },
  {
    "passThroughId": "Wrong Integer Type Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "What did you buy on black friday shopping? Sears Sucks big time...This is what I want to say about the message body and test for Author Field ",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 57,
    "languageTag": "en"
  },
  {
    "passThroughId": "Wrong Type Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Target had good deals on black friday shoppping and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 65,
    "languageTag": "en"
  },
  {
    "passThroughId": "None Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Black friday is gone and the weather was good for shopping. Body shop is close by and I want to pass this message and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 766,
    "languageTag": "en"
  },
  {
    "passThroughId": "Empty Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "This is good for the community I think. There are really good things that is being done. This message is for everybody and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://good.things.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 78,
    "languageTag": "en"
  },
  {
    "passThroughId": "Missing Message parentId",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Christmas is soon approaching.. holdiay shopping is done on black friday and I hope the weather will be nice and test for Author Field. This week the weather is very cold. Whole body is shivering, the message is not right",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "languageTag": "en"
  },
  {
    "passThroughId": "Duplicate Message parentId (1)",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Christmas is soon approaching.. holdiay shopping is done on black friday and I hope the weather will be nice and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://example.acne.com/messages/jsmith/1234567890",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 159677303404,
    "languageTag": "en"
  },
  {
    "passThroughId": "Duplicate Message parentId (2)",
    "author": {
      "displayName": "jsmith",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://example.acne.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://example.acne.com",
    "texts": [
      {
        "text": "example subject",
        "name": "subject"
      },
      {
        "text": "Christmas is soon approaching.. holdiay shopping is done on black friday and I hope the weather will be nice and test for Author Field",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://christmas.shopping.com/messages/psridhara/123",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 159677303404,
    "languageTag": "en"
  },
  {
    "passThroughId": "Testing EDC End to End",
    "author": {
      "displayName": "psridhara",
      "kloutScore": 83,
      "passThroughId": "example author",
      "name": "Jonh Smith",
      "followersCount": 205,
      "gender": "male",
      "imageUrl": "http://example.acne.com/profiles/jsmith/picture.jpg",
      "birthDate": "1990-03-20T17:47:46.000Z",
      "sourceMessagesCount": 1780,
      "profileUrl": "http://ede.test.com/profiles/jsmith",
      "utcOffset": "+08:00",
      "location": {
        "displayName": "USA",
        "locality": "San Jose",
        "country": "us",
        "region": "US-CA",
        "longitude": 33.05,
        "subRegion": "Main Street",
        "latitude": 37.33
      },
      "friendsCount": 55,
      "email": "jsmith@example.acne.com",
      "createdAt": "2015-06-12T16:30:03.000Z",
      "description": "good user"
    },
    "url": "http://ede.test.com",
    "texts": [
      {
        "text": "EDC Test",
        "name": "subject"
      },
      {
        "text": "I hope December doesnt end. This is a very nice month. People are in vacation mood and test for Author Field and so is PReethi on June 12th",
        "name": "body"
      }
    ],
    "publishedAt": "2015-06-12T16:38:07.000Z",
    "source": {
      "url": "http://christmas.shopping.com/messages/psridhara/123",
      "name": "example source"
    },
    "location": {
      "displayName": "USA",
      "locality": "San Jose",
      "country": "us",
      "region": "US-CA",
      "longitude": 33.05,
      "subRegion": "Main Street",
      "latitude": 37.33
    },
    "parentId": 159677303404,
    "languageTag": "en"
  }
]

Custom Data Source Analyzer API

What can this API do?

Audience

SE&M & CDSA API Licenses

License Keys

Custom Topics

Sample Errors Found Response

Success Response

Failure Response

Standard Response:

Successful Validation Response

Errors Found Response

Standard Response:

Standard Response:

Standard Response:

Related Topics