Map/Reduce Terminology

Term	Definition	More Information
buffer size	An option on the script deployment record. This choice determines the number of key-value pairs that a map or reduce job can process before information about the job’s progress is saved to the database. As a general best practice, leave this value set to 1.	Buffer Size
deployment instance	A task created to process a script deployment. A deployment instance is created when the deployment is submitted for processing. Only one unfinished instance of a particular script deployment can exist at any time.	Map/Reduce Script Submission
exitOnError	A configuration option that affects the behavior of a script when an uncaught error interrupts a map or reduce function. When exitOnError is set to true, and all available retries have been used, the script exits the current stage and goes to the summarize stage.	exitOnError
function invocation	An execution of a function. Some map/reduce entry point functions, such as the getInputData and summarize functions, are invoked only one time during the script’s processing. Others, such as the map and reduce functions, are invoked multiple times. For example, if the getInputData stage provides 20 key-value pairs to the map stage, then the map function is invoked 20 times, one time for each pair.	SuiteScript 2.x Map/Reduce Script Type
getInputData	The first stage in the processing of a map/reduce script. During this stage, your script must return an object that can be transformed into a list of key-value pairs. You use the getInputData entry point to identify the function that executes during this stage.	Map/Reduce Script Stages
governance	A system of rules governing your usage of NetSuite. Specific limits exist for every script type, including map/reduce.	Map/Reduce Governance SuiteScript Governance and Limits
hard limit	A type of map/reduce governance limit that, when violated, causes an interruption to the current function invocation. Make sure you follow the Map/Reduce Script Best Practices section in the SuiteScript Developer Guide to avoid problems with hard limits.	Map/Reduce Governance Map/Reduce Script Best Practices
job	A unit of execution managed by SuiteCloud Processors. The processing of a map/reduce script is always handled by multiple jobs. At least one job is created to handle the processing of each stage.	Map/Reduce Script Stages SuiteCloud Processors
map	The second stage in the processing of a map/reduce script. A map function is optional, but your script must use either a map or reduce function (or both). When used, a map function processes data provided by the getInputData stage. The map function is invoked one time for each key-value pair that is provided.	Map/Reduce Script Stages map(mapContext)
map/reduce	A computing paradigm designed for the processing of large data sets.	https://en.wikipedia.org/wiki/MapReduce
map/reduce script type	A SuiteScript 2.x script type based on the map/reduce paradigm.	Map/Reduce Key Concepts
parallel stage	A type of stage that can be handled by multiple jobs that work simultaneously. The map and reduce stages are parallel stages. See also serial stage.	Map/Reduce Script Stages
priority	A property of a job. Job priority determines the order in which the scheduler sends jobs to the processor pool. Priorities can be set on the deployment record or on the SuiteCloud Processors Priority Settings Page.	Map/Reduce Script Deployment Record SuiteCloud Processors Priority Levels
processor	In the context of SuiteCloud Processors, a virtual unit of processing power that executes a job. A processor is not an individual physical entity but rather a single processing thread.	Concurrency Limit SuiteCloud Processors
processor pool	In the context of SuiteCloud Processors, the total processors available to your NetSuite account. The number of processors available varies depending on your licensing. The number of processors available to process a particular script is determined by the value you choose for the Concurrency Limit field on the script deployment record.	Concurrency Limit SuiteCloud Processors
reduce	The fourth stage in the processing of a map/reduce script. A reduce function is optional, but your script must use either a map or reduce function (or both). When used, a reduce function processes data provided by the getInputData stage or, if a map function is used, by the map stage. The reduce function is invoked one time for each unique key from the list of all key-value pairs provided. The value for each of the unique keys passed to the reduce function is an array of all values with that key.	Map/Reduce Script Stages reduce(reduceContext)
retryCount	A configuration option that is relevant when a map or reduce function is interrupted by an uncaught error or by an application server restart. This setting determines whether the system invokes the map or reduce function again for any key-value pairs that were left in an uncertain state following the interruption.	retryCount
script deployment	A set of configuration choices for how to execute a script. These choices are stored on a script deployment record. To execute a map/reduce script, you must create a deployment record and submit the deployment.	Map/Reduce Script Submission
serial stage	A type of stage that is processed in its entirety by only one job. The getInputData, shuffle, and summarize stages are serial stages.	Map/Reduce Script Stages
shuffle	The third stage in the processing of a map/reduce script. This stage is significant if your script uses a reduce function. In these cases, the shuffle stage sorts data provided by the getInputData stage or, if a map function is used, by the map stage. In either case, the shuffle stage creates a set of key-value pairs where each key is unique, and each value is an array. These pairs are passed to the reduce stage for further processing.	Map/Reduce Script Stages
soft limit	A type of map/reduce governance rule that, after it is surpassed, causes a map or reduce job to automatically yield. When the job yields, its work is rescheduled. Progress toward a soft limit is checked only after a function invocation has completed. Therefore this limit never causes an interruption to the function invocation.	Map/Reduce Governance
SuiteCloud Processors	Processes map/reduce and scheduled scripts.	SuiteCloud Processors
summarize	The final stage in the processing of a map/reduce script. You can use this stage to log data about the work of the script.	Map/Reduce Script Stages summarize(summaryContext)
task	The set of work that represents a map/reduce deployment that has been submitted for processing. Each task is executed by a minimum of five jobs, one for each stage.	Map/Reduce Script Submission
yielding	A behavior that allows a map/reduce script to manage some aspects of its own resource consumption. After a map/reduce job has surpassed a soft limit, it automatically yields, and its work is rescheduled for later. Yielding never interrupts a function invocation. Note also that this process is automatic. SuiteScript 2.x does not contain an API that lets you force a script to yield.	Yield After Minutes

Term

Definition

More Information

buffer size

An option on the script deployment record. This choice determines the number of key-value pairs that a map or reduce job can process before information about the job’s progress is saved to the database. As a general best practice, leave this value set to 1.

Buffer Size

deployment instance

A task created to process a script deployment. A deployment instance is created when the deployment is submitted for processing. Only one unfinished instance of a particular script deployment can exist at any time.

Map/Reduce Script Submission

exitOnError

A configuration option that affects the behavior of a script when an uncaught error interrupts a map or reduce function. When exitOnError is set to true, and all available retries have been used, the script exits the current stage and goes to the summarize stage.

exitOnError

function invocation

An execution of a function. Some map/reduce entry point functions, such as the getInputData and summarize functions, are invoked only one time during the script’s processing. Others, such as the map and reduce functions, are invoked multiple times. For example, if the getInputData stage provides 20 key-value pairs to the map stage, then the map function is invoked 20 times, one time for each pair.

SuiteScript 2.x Map/Reduce Script Type

getInputData

The first stage in the processing of a map/reduce script. During this stage, your script must return an object that can be transformed into a list of key-value pairs. You use the getInputData entry point to identify the function that executes during this stage.

Map/Reduce Script Stages

governance

A system of rules governing your usage of NetSuite. Specific limits exist for every script type, including map/reduce.

hard limit

A type of map/reduce governance limit that, when violated, causes an interruption to the current function invocation. Make sure you follow the Map/Reduce Script Best Practices section in the SuiteScript Developer Guide to avoid problems with hard limits.

job

A unit of execution managed by SuiteCloud Processors. The processing of a map/reduce script is always handled by multiple jobs. At least one job is created to handle the processing of each stage.

map

The second stage in the processing of a map/reduce script. A map function is optional, but your script must use either a map or reduce function (or both). When used, a map function processes data provided by the getInputData stage. The map function is invoked one time for each key-value pair that is provided.

map/reduce

A computing paradigm designed for the processing of large data sets.

https://en.wikipedia.org/wiki/MapReduce

map/reduce script type

A SuiteScript 2.x script type based on the map/reduce paradigm.

Map/Reduce Key Concepts

parallel stage

A type of stage that can be handled by multiple jobs that work simultaneously. The map and reduce stages are parallel stages. See also serial stage.

Map/Reduce Script Stages

priority

A property of a job. Job priority determines the order in which the scheduler sends jobs to the processor pool. Priorities can be set on the deployment record or on the SuiteCloud Processors Priority Settings Page.

processor

In the context of SuiteCloud Processors, a virtual unit of processing power that executes a job. A processor is not an individual physical entity but rather a single processing thread.

processor pool

In the context of SuiteCloud Processors, the total processors available to your NetSuite account. The number of processors available varies depending on your licensing. The number of processors available to process a particular script is determined by the value you choose for the Concurrency Limit field on the script deployment record.

reduce

The fourth stage in the processing of a map/reduce script. A reduce function is optional, but your script must use either a map or reduce function (or both). When used, a reduce function processes data provided by the getInputData stage or, if a map function is used, by the map stage. The reduce function is invoked one time for each unique key from the list of all key-value pairs provided. The value for each of the unique keys passed to the reduce function is an array of all values with that key.

retryCount

A configuration option that is relevant when a map or reduce function is interrupted by an uncaught error or by an application server restart. This setting determines whether the system invokes the map or reduce function again for any key-value pairs that were left in an uncertain state following the interruption.

retryCount

script deployment

A set of configuration choices for how to execute a script. These choices are stored on a script deployment record. To execute a map/reduce script, you must create a deployment record and submit the deployment.

Map/Reduce Script Submission

serial stage

A type of stage that is processed in its entirety by only one job. The getInputData, shuffle, and summarize stages are serial stages.

Map/Reduce Script Stages

shuffle

The third stage in the processing of a map/reduce script. This stage is significant if your script uses a reduce function. In these cases, the shuffle stage sorts data provided by the getInputData stage or, if a map function is used, by the map stage. In either case, the shuffle stage creates a set of key-value pairs where each key is unique, and each value is an array. These pairs are passed to the reduce stage for further processing.

Map/Reduce Script Stages

soft limit

A type of map/reduce governance rule that, after it is surpassed, causes a map or reduce job to automatically yield. When the job yields, its work is rescheduled. Progress toward a soft limit is checked only after a function invocation has completed. Therefore this limit never causes an interruption to the function invocation.

Map/Reduce Governance

SuiteCloud Processors

Processes map/reduce and scheduled scripts.

SuiteCloud Processors

summarize

The final stage in the processing of a map/reduce script. You can use this stage to log data about the work of the script.

task

The set of work that represents a map/reduce deployment that has been submitted for processing. Each task is executed by a minimum of five jobs, one for each stage.

Map/Reduce Script Submission

yielding

A behavior that allows a map/reduce script to manage some aspects of its own resource consumption. After a map/reduce job has surpassed a soft limit, it automatically yields, and its work is rescheduled for later. Yielding never interrupts a function invocation. Note also that this process is automatic. SuiteScript 2.x does not contain an API that lets you force a script to yield.

Yield After Minutes

Related Topics