Transform Service log

The Transform Service logs are stored in the $BDD_HOME/logs/transformservice directory. When the Transform Service receives a request to preview a data set, it logs the schema of that data set, as shown in this abbreviated example:

16/06/29 14:51:29.775 - INFO  [GridPreviewRunner@37]:- Start processing preview request
for MdexCollectionIdentifier{databaseName=edp_cli_edp_4dd5ac28-2e85-4efc-a3c2-391b6a78f69c, 
collectionName=edp_cli_edp_4dd5ac28-2e85-4efc-a3c2-391b6a78f69c}
16/06/29 14:51:29.778 - INFO  [GridPreviewRunner@38]:- class TransformConfig {
    schema: [class Column {
        name: production_country
        type: STRING
        isSingleAssign: true
        isRecordSearchable: false
        isValueSearchable: true
        language: en
    }, class Column {
        name: dealer_geocode
        type: GEOCODE
        isSingleAssign: true
        isRecordSearchable: false
        isValueSearchable: false
        language: en
...
    }, class Column {
        name: labor_description
        type: STRING
        isSingleAssign: true
        isRecordSearchable: false
        isValueSearchable: true
        language: en
    }]
    transformList: [class PutColumnTransform {
        class TransformInfo {
            transformType: null
        }
        column: class Column {
            name: sentiments
            type: STRING
            isSingleAssign: true
            isRecordSearchable: null
            isValueSearchable: null
            language: null
        }
        exceptionAction: class SetNullAction {
            class TransformExceptionAction {
                actionType: null
            }
            actionType: null
        }
        transformType: null
        script: getSentiment(complaint)
    }]
    resultRowCount: 50
    sort: null
    filter: null
    databaseName: edp_cli_edp_4dd5ac28-2e85-4efc-a3c2-391b6a78f69c
    collectionName: edp_cli_edp_4dd5ac28-2e85-4efc-a3c2-391b6a78f69c
    optimization: null

When it receives its first preview or transformation request, the Transform Service logs the system, Spark, and Hadoop configurations that it is using. An abbreviated configuration entry is as follows:

Number of processors available: 2
Total available memory: 226 MB
Free memory: 170 MB
Maximum available memory: 3403 MB

16/06/29 14:51:37.807 - INFO  [LocalSparkClient@50]:- Spark configuration:
spark.externalBlockStore.folderName = spark-78c17408-b81f-4d0e-a4ac-f06174e67c42
spark.driver.cores = 4
spark.io.compression.codec = lzf
spark.lib = /localdisk/Oracle/Middleware/BDD-1.3.0.35.999/transformservice/bddservices/spark_lib/spark-assembly.jar
spark.app.name = transformservice
spark.executor.memory = 4g
spark.master = local[8]
spark.driver.host = 10.152.105.219
spark.executor.id = driver
spark.app.id = local-1467226296747
spark.driver.port = 50018
spark.local.dir = /localdisk/Oracle/Middleware/BDD-1.3.0.35.999/transformservice/tmp
spark.fileserver.uri = http://10.152.105.219:13880
spark.ui.enabled = false
spark.driver.maxResultSize = 4g

16/06/29 14:51:37.966 - INFO  [LocalSparkClient@59]:- Hadoop configuration:
s3.replication = 3
mapreduce.output.fileoutputformat.compress.type = BLOCK
mapreduce.jobtracker.jobhistory.lru.cache.size = 5
hadoop.http.filter.initializers = org.apache.hadoop.http.lib.StaticUserWebFilter
...
yarn.resourcemanager.system-metrics-publisher.enabled = false
mapreduce.client.output.filter = FAILED