Logging Errors

With any map/reduce script, you should include logic that checks for errors that may have occurred during the getInputData, map, and reduce stages. You can access data about errors using the context objects made available to each entry point. Use the properties shown in the following table.

Stage where error occurred	Property that contains data
Get Input Data	inputSummary.error
Map	reduceContext.errors — Contains the error codes recorded during previous attempts to process the current key-value pair. mapSummary.errors — Contains all error codes recorded during the map stage.
Reduce	reduceContext.errors — Contains the error codes recorded during previous attempts to process the current key-value pair. reduceSummary.errors — Contains all error codes recorded during the reduce stage.

Syntax

The following snippets shows how you can capture data about errors in various stages.

Map Stage

You can use this snippet in a map function. The snippet logs data only if an error was thrown during a previous invocation of the map function for the same key-value pair currently being processed.

          // Create a log entry showing each full serialized error thrown
// during previous attempts to process the current key-value pair.

mapContext.errors.iterator().each(function (key, error, executionNo){
    
    log.error({
        title:  'Map error for key: ' + key + ', execution no  ' + executionNo,
        details: error
    });
    
    return true;
}); 

        

Reduce Stage

You can use this snippet in a reduce function. The snippet logs data only if an error was thrown during a previous invocation of the reduce function for the same key-value pair currently being processed.

          // Create a log entry showing each full serialized error thrown
// during previous attempts to process the current key-value pair.

reduceContext.errors.iterator().each(function (key, error, executionNo){
    log.error({
        title:  'Reduce error for key: ' + key + ', execution no  ' + executionNo,
        details: error
    });
    return true;
}); 

        

Summarize Stage

You can use these snippets in a summarize function. They log data about errors thrown during previous stages.

          // If an error was thrown during the input stage, log the error. 

if (summary.inputSummary.error)
    {
        log.error({
            title: 'Input Error', 
            details: summary.inputSummary.error)
    });


// For each error thrown during the map stage, log the error, the corresponding key,
// and the execution number. The execution number indicates whether the error was 
// thrown during the the first attempt to process the key, or during a
// subsequent attempt.

summary.mapSummary.errors.iterator().each(function (key, error, executionNo){
    log.error({
        title: 'Map error for key: ' + key + ', execution no.  ' + executionNo,
        details: error
    });
    return true;
});


// For each error thrown during the reduce stage, log the error, the corresponding
// key, and the execution number. The execution number indicates whether the error was 
// thrown during the the first attempt to process the key, or during a
// subsequent attempt.

summary.reduceSummary.errors.iterator().each(function (key, error, executionNo){
    log.error({
           title: 'Reduce error for key: ' + key + ', execution no. ' + executionNo,
           details: error
       });
       return true;
   }); 

        

Related Topics

Map/Reduce Script Error Handling

System Response After a Map/Reduce Interruption

Configuration Options for Handling Map/Reduce Interruptions

Execution of Restarted Map/Reduce Stages

Adding Logic to Handle Map/Reduce Restarts

General Notices