Map/Reduce Script Deployment Record

Before a map/reduce script can be executed, you must create at least one deployment record for the script.

The deployment record for a map/reduce script is similar to that of other script types. However, a map/reduce script deployment contains some additional fields. Some of these fields are specific to SuiteCloud Processors, which are used to execute map/reduce scripts. Others are specific to map/reduce features. This topic describes all of the available fields.

You can access a map/reduce script deployment record in the following ways:

To open an existing deployment record for editing, go to Customization > Scripting > Script Deployments. Locate the appropriate record, and click Edit.
To start creating a new deployment record, open the appropriate script record in view mode, and click the Deploy Script button. For help creating a script record, see Script Record Creation.

Body Fields

The following table summarizes the fields available on the map/reduce script deployment record. Note that some fields are available only when you edit or view an existing deployment record.

Field	Description
Script	A link to the script record associated with the deployment. This value cannot be changed, even on a new deployment record. If you begin the process of creating a deployment and realize that you selected the wrong script record, you must start the process over.
Title	The user-defined name for the deployment.
ID	A unique identifier for the deployment. On a new record, you can customize this identifier by entering a value in the ID field. You should customize the ID if you plan to bundle the deployment for installation into another account, because using custom IDs helps avoid naming conflicts. IDs must be lowercase and cannot use spaces. You can use an underscore as a delineator. If you do not enter a value, the system automatically generates one. In both cases, the system automatically adds the prefix `customdeploy` to the ID created when the record is saved. Although not preferred, you can change the ID on an existing deployment by clicking the Change ID button.
Deployed	A configuration option that indicates whether the deployment is active. This box must be checked if you want the script to execute. Otherwise, the system uses the following behavior: When the Deployed box is cleared, and the Status is Not Scheduled, the Save and Execute option is no longer available, and the deployment cannot be submitted programmatically. When the Deployed box is cleared, and the deployment record’s Status is Scheduled, any times configured on the Schedule subtab are ignored and the script deployment is not submitted.
Status	A value that determines how and when a script deployment can be submitted for processing. The primary options are: Scheduled —The script deployment is submitted for processing at the times indicated on the Schedule subtab. Not Scheduled — The script deployment is submitted for processing only when invoked manually, either through the UI or programmatically. Note also that, regardless of the Status, the system submits the deployment for processing only if the Deployed box is checked. For more details on this choice, see Status.
See Instances	A link to the Map/Reduce Script Status Page, filtered for all instances of this deployment record, for the current day. You can change the filtering options if needed. For details on working with the Map/Reduce Script Status page, see Map/Reduce Script Status Page.
Log Level	A value that determines what type of log messages are displayed on the Execution Log of both the deployment record and associated script record. The available levels are: Debug — suitable for scripts still being tested; this level shows all debug, audit, error and emergency messages. Use Audit, Error, or Emergency — suitable for scripts in production mode. Of these three, Audit is the most inclusive and Emergency the most exclusive. For more details on each level, see Log Level.
Execute As Role	Indicates the role used to run the script. For map/reduce deployments, this value is automatically set to Administrator and cannot be edited.
Priority	A measure of how urgently this script should be processed relative to other map/reduce and scheduled scripts that have been submitted. This value is assigned to each job associated with the script deployment. The priority affects when the SuiteCloud Processors scheduler sends these jobs to the processor pool. For more details, see Priority. Important: You must understand SuiteCloud Processors before you change this setting. For details, see SuiteCloud Processors Priority Levels.
Concurrency Limit	Determines the number of SuiteCloud Processors that can be used to process the jobs associated with the script deployment. For more details on this field, see Concurrency Limit.
Submit All Stages At Once	Determines whether the system creates jobs for all of the map/reduce stages simultaneously when the script deployment is submitted for processing. In general, clear this field only if the script deployment is relatively low in priority. For more details on this field, see Submit All Stages At Once.
Yield After Minutes	A soft time limit on how long the script deployment’s map and reduce jobs may run before yielding. You can enter a number from 3 to 60. In general, each time a map or reduce job finishes a function invocation, the system checks to see whether this time limit has been exceeded. If it has, the job yields so that other jobs can be processed. A new job is created to take on the work that was being processed by the job that yielded. For more details on this field, see Yield After Minutes.
Buffer Size	A value that indicates how many key-value pairs a map or reduce job can process before information about the job’s progress is saved to the database. A low Buffer Size value minimizes the risk of any pairs being processed twice, which can lead to data duplication. In general, leave this value set to 1 unless you have special circumstances that dictate otherwise. For more details on this field, see Buffer Size.

Field

Description

Script

A link to the script record associated with the deployment. This value cannot be changed, even on a new deployment record. If you begin the process of creating a deployment and realize that you selected the wrong script record, you must start the process over.

Title

The user-defined name for the deployment.

A unique identifier for the deployment.

On a new record, you can customize this identifier by entering a value in the ID field. You should customize the ID if you plan to bundle the deployment for installation into another account, because using custom IDs helps avoid naming conflicts. IDs must be lowercase and cannot use spaces. You can use an underscore as a delineator.

If you do not enter a value, the system automatically generates one. In both cases, the system automatically adds the prefix customdeploy to the ID created when the record is saved.

Although not preferred, you can change the ID on an existing deployment by clicking the Change ID button.

Deployed

A configuration option that indicates whether the deployment is active. This box must be checked if you want the script to execute. Otherwise, the system uses the following behavior:

When the Deployed box is cleared, and the Status is Not Scheduled, the Save and Execute option is no longer available, and the deployment cannot be submitted programmatically.
When the Deployed box is cleared, and the deployment record’s Status is Scheduled, any times configured on the Schedule subtab are ignored and the script deployment is not submitted.

Status

A value that determines how and when a script deployment can be submitted for processing. The primary options are:

Scheduled —The script deployment is submitted for processing at the times indicated on the Schedule subtab.
Not Scheduled — The script deployment is submitted for processing only when invoked manually, either through the UI or programmatically.

Note also that, regardless of the Status, the system submits the deployment for processing only if the Deployed box is checked.

For more details on this choice, see Status.

See Instances

A link to the Map/Reduce Script Status Page, filtered for all instances of this deployment record, for the current day. You can change the filtering options if needed.

For details on working with the Map/Reduce Script Status page, see Map/Reduce Script Status Page.

Log Level

A value that determines what type of log messages are displayed on the Execution Log of both the deployment record and associated script record. The available levels are:

Debug — suitable for scripts still being tested; this level shows all debug, audit, error and emergency messages.
Use Audit, Error, or Emergency — suitable for scripts in production mode. Of these three, Audit is the most inclusive and Emergency the most exclusive.

For more details on each level, see Log Level.

Execute As Role

Indicates the role used to run the script. For map/reduce deployments, this value is automatically set to Administrator and cannot be edited.

Priority

A measure of how urgently this script should be processed relative to other map/reduce and scheduled scripts that have been submitted. This value is assigned to each job associated with the script deployment. The priority affects when the SuiteCloud Processors scheduler sends these jobs to the processor pool. For more details, see Priority.

Important:

You must understand SuiteCloud Processors before you change this setting. For details, see SuiteCloud Processors Priority Levels.

Concurrency Limit

Determines the number of SuiteCloud Processors that can be used to process the jobs associated with the script deployment. For more details on this field, see Concurrency Limit.

Submit All Stages At Once

Determines whether the system creates jobs for all of the map/reduce stages simultaneously when the script deployment is submitted for processing. In general, clear this field only if the script deployment is relatively low in priority. For more details on this field, see Submit All Stages At Once.

Yield After Minutes

A soft time limit on how long the script deployment’s map and reduce jobs may run before yielding. You can enter a number from 3 to 60. In general, each time a map or reduce job finishes a function invocation, the system checks to see whether this time limit has been exceeded. If it has, the job yields so that other jobs can be processed. A new job is created to take on the work that was being processed by the job that yielded. For more details on this field, see Yield After Minutes.

Buffer Size

A value that indicates how many key-value pairs a map or reduce job can process before information about the job’s progress is saved to the database. A low Buffer Size value minimizes the risk of any pairs being processed twice, which can lead to data duplication. In general, leave this value set to 1 unless you have special circumstances that dictate otherwise. For more details on this field, see Buffer Size.

Status

The Status field determines how and when the script deployment may be submitted for processing. The default value is Not Scheduled.

Regardless of how the script deployment is submitted, it does not necessarily execute at the exact time scheduled, or at the exact time that it is manually invoked. There may be a short system delay, even if no scripts are before it. If there are scripts already waiting to be executed, the script may need to wait until others have completed. For details on this behavior, see SuiteCloud Processors.

Scheduled

When a deployment’s Status is set to Scheduled, the script deployment is submitted for processing according to a one-time or recurring schedule. You define this schedule by using the deployment record’s Schedule Subtab. With this approach, after you save the deployment, you do not have to take any other steps for the deployment to be submitted for processing.

Note also:

If you schedule a recurring submission with an end date, or a one-time submission, the deployment record’s status remains Scheduled even after the script completes its execution.
You cannot submit an on-demand instance of the script deployment when it has a status of Scheduled.

Not Scheduled

When a deployment’s Status is set to Not Scheduled, the deployment is available to be submitted on an on-demand basis. If you want the deployment to be submitted for processing, you must manually submit it, either through the NetSuite UI or programmatically. You can use either of the following:

The Save and Execute option on the deployment record. See also Submitting an On-Demand Map/Reduce Script Deployment from the UI.
The task.ScheduledScriptTask API. See also Submitting an On–Demand Map/Reduce Script Deployment from a Script.

You can submit the script deployment only if there is no other unfinished instance of the same script deployment. If you want multiple instances of the script to be submitted for processing simultaneously, you must create multiple deployment records for the script record. For details, see Submitting Multiple Deployments of the Same Script.

Testing

When a deployment’s Status is set to Testing, only the script owner will be able to test and debug the script without submitting for processing. You have several options available to test and debug your Map/Reduce script. For more information about Map/Reduce Script testing, see Map/Reduce Script Testing and Troubleshooting.

Log Level

The Log Level field determines what type of log messages are displayed in the Execution Log.

In general, if a script is still being tested, use the Debug log level. This option includes more messages than the other log levels, including messages created by log.debug(options), log.audit(options), log.error(options), and log.emergency(options).

If a script is in production, use one of the following levels:

Audit — This level includes a record of events that have occurred during the processing of the script (for example, “A request was made to an external site”). This level includes log messages created by log.audit(options), log.error(options), and log.emergency(options).
Error — A log level set to Error shows only unexpected script errors, including log messages created by log.error(options) and log.emergency(options).
Emergency — Includes only the most critical messages, including log messages created by log.emergency(options).

The default value is Debug.

Priority

If multiple map/reduce and scheduled script deployments are submitted for processing at the same time, some scripts may have to wait to be processed. To handle this type of situation, you can use the Priority field. This setting determines how quickly the script deployment instance, and each of its jobs, should be sent for processing relative to other script deployment instances that were created at the same time. The priority for the deployment is applied to each of the deployment instance’s jobs.

The choices are as follows:

High — Use to mark critical deployments that require more immediate processing. The scheduler sends high-priority jobs to the processor pool first.
Standard — This is the default setting. It is considered to be a medium priority level. The scheduler sends medium–priority jobs to the processor pool if there are no high-priority jobs waiting.
Low — Use to mark deployments that can tolerate a longer wait time. The scheduler sends low-priority jobs to the processor pool if there are no high– or standard-priority jobs waiting.

Important:

You must understand SuiteCloud Processors before you change this setting. See SuiteCloud Processors Priority Levels.

Concurrency Limit

The map/reduce script type permits parallel processing. With parallel processing, multiple SuiteCloud Processors can work simultaneously to execute a single script deployment instance. You can control the number of processors used for each script instance by using the Concurrency Limit field on the script deployment record.

Note:

This setting affects the map and reduce stages only. These stages are the only ones that permit parallel processing.

For example, if you specify a concurrency limit of 5, the system creates five map jobs and five reduce jobs. If you do not specify a limit, the maximum number of processors available to your account is used. The default value is 2. For more information, see SuiteCloud Processors Processor Allotment Per Account.

If you use a SuiteCloud project to modify the concurrency limit in a script deployment record, SDF adjusts the concurrency limit automatically if you specify a value that exceeds the limit available in the target account. For example, if you specify a concurrency limit of 10 and deploy your SuiteCloud project to an account with a limit of 5, the Concurrency Limit field on the script deployment record in the account is set to 5. The XML representation of the map/reduce script remains at the original value of 10. For more information, see Setting a Concurrency Limit on Your Map/Reduce Script Deployment in SDF.

Note:

The Concurrency Limit field was introduced in 2017.2, as part of the SuiteCloud Processors feature. If you are editing a deployment record that was created prior to 2017.2, be aware that when your account was upgraded, the Concurrency Value field was initially set to a value that corresponds to the number of queues that had been saved for the Queues field.

Submit All Stages At Once

Every map/reduce script deployment instance is processed by multiple jobs. At least one job is created for each stage used by the script. Every map/reduce script must use either four or five stages: getInputData, shuffle, summarize, and either map or reduce (or both). However, the jobs for the various stages are not necessarily submitted at the same time. This behavior is controlled by the Submit All Stages at Once option.

Map/reduce stages must occur in a specific sequence. When the Submit All Stages at Once option is disabled, the system waits to submit the jobs for each stage until after the prerequisite job completes.

In contrast, when the Submit All Stages at Once option is enabled, the system submits jobs for all stages simultaneously. This behavior increases the likelihood that all jobs associated with the script deployment instance finish, without gaps, before another script begins executing. However, be aware that this option does not guarantee that no gaps occur. For example, because a map/reduce job can yield, a long-running job may be forced to end, and a job associated with another script may begin executing in its place. For these reasons, you should not rely on this option if you need to enforce a strict execution sequence among scripts. The only way to enforce a strict sequence is to have one script schedule another during the summarize stage. A script can be scheduled programmatically by using the task.create(options) method. For more details, see Submitting an On–Demand Map/Reduce Script Deployment from a Script.

The Submit All Stages at Once option is enabled by default. In general, you should leave this option enabled.

Yield After Minutes

The Yield After Minutes field helps you prevent any processor from being monopolized by a long-running map or reduce job.

Here’s how this setting works: During the map and reduce stages, after each function invocation, the system checks to see how long the map or reduce job has been running. If the amount of elapsed time has surpassed the number of minutes identified in the Yield After Minutes field, the job gracefully ends its execution, and a new job is created to take its place. The new job has the same priority as the old one, but a later timestamp. This behavior is known as yielding.

By default, Yield After Minutes is set to 60, but you can enter any number from 3 to 60.

The system never interrupts a function invocation for this limit. Also, the system never ends a map or reduce job before the limit has been reached, but only after it has been passed. For that reason, the degree to which the limit is surpassed varies depending on the duration of your function invocation. For example, if the Yield After Minutes limit is 3 minutes, but your function takes 15 minutes to complete, then in practice the job yields after 15 minutes, not 3 minutes.

Yielding is also affected by a governance limit. This limit is 10,000 usage units for each map and reduce job. This limit works in the same way as the Yield After Minutes limit: The system waits until after each function invocation ends to determine whether the usage-unit limit has been surpassed. If it has, the job yields, even if the Yield After Minutes limit has not been exceeded.

Buffer Size

The purpose of the Buffer Size field is to control how frequently a map or reduce job saves data about its progress. In general, you should leave this field set to 1.

To understand this setting, remember how map and reduce jobs work: First, the job flags some number of key-value pairs that require processing. Then it processes the pairs that it flagged. Then it saves data about the work it did. This data includes information about which key-value pairs were processed, how many usage points were consumed, and so on. The process repeats either until the job yields or until no key-value pairs remain.

The Buffer Size field controls how many pairs are flagged at one time. So if you leave this field set to its default of 1, the job flags one pair, processes it, and saves the data. Then it repeats the cycle.

You can set Buffer Size to any of the following values: 1, 2, 4, 8, 16, 32, or 64. However, the disadvantage of choosing a higher number is that, if the job is interrupted by an application server restart, there is a greater likelihood that one or more key-value pairs will be processed twice. On the other hand, setting this field to a higher number can be more efficient in certain cases.

Use the following guidance:

In general, leave this value set to 1, particularly if the script is processing records.
You can choose a higher buffer size value if the script is performing fast, algorithmic operations, or if other special circumstances dictate that you deviate from the default setting.

Schedule Subtab

The following table summarizes the fields on the Schedule subtab. These settings are honored only if the deployment record’s Status is set to Scheduled and the Deployed box is checked.

Field	Description
Single Event	The map/reduce script deployment is submitted only one time. Use this option if you want to schedule a future one-time submission.
Daily Event	The map/reduce script deployment is submitted every x number of days. If you schedule the submission to recur every x minutes or hours, the schedule starts over on the next scheduled day. For example, your deployment is set to submit daily, starting at 3:00 am and recurring every five hours. A scheduled script instance is submitted at 3:00 am, 8:00 am, 1:00 pm, 6:00 pm, and 11:00 pm. At midnight, the schedule starts over and the next submission is at 3:00 am.
Weekly Event	The map/reduce script deployment is submitted at least one time per scheduled week. If you schedule the submission to recur every x minutes or hours, the schedule starts over on the next scheduled day. For example, your deployment is set to submit on Tuesday and Wednesday, starting at 3:00 am and recurring every five hours. The deployment is submitted on Tuesday at 3:00 am, 8:00 am, 1:00 pm, 6:00 pm, and 11:00 pm. On Wednesday, the schedule starts over and the next submission is at 3:00 am.
Monthly Event	The map/reduce script deployment is submitted at least one time per month.
Yearly Event	The scheduled script deployment is submitted at least one time per year.
Start Date	The first submission occurs on this date. This field is required if a one-time or recurring schedule is set.
Start Time	If a value is selected, the first submission occurs at the time specified.
Repeat	If a value is selected, the first submission occurs on the date and time selected. A new script deployment instance is then created and submitted every x minutes or hours until the end of the start date. If applicable, the schedule starts over on the next scheduled day. For example, your deployment is set to submit on Tuesday and Wednesday, starting at 3:00 am and recurring every five hours. The deployment is submitted on Tuesday at 3:00 am, 8:00 am, 1:00 pm, 6:00 pm, and 11:00 pm. On Wednesday, the schedule starts over and the next submission is at 3:00 am.
End By	If a value is entered, the last submission ends by this date. If you schedule the submission to recur every x minutes or hours, a new script deployment instance is created and submitted every x minutes or hours until the end date.
No End Date	The schedule does not have a set end date.

Field

Description

Single Event

The map/reduce script deployment is submitted only one time. Use this option if you want to schedule a future one-time submission.

Daily Event

The map/reduce script deployment is submitted every x number of days. If you schedule the submission to recur every x minutes or hours, the schedule starts over on the next scheduled day.

For example, your deployment is set to submit daily, starting at 3:00 am and recurring every five hours. A scheduled script instance is submitted at 3:00 am, 8:00 am, 1:00 pm, 6:00 pm, and 11:00 pm. At midnight, the schedule starts over and the next submission is at 3:00 am.

Weekly Event

The map/reduce script deployment is submitted at least one time per scheduled week. If you schedule the submission to recur every x minutes or hours, the schedule starts over on the next scheduled day.

For example, your deployment is set to submit on Tuesday and Wednesday, starting at 3:00 am and recurring every five hours. The deployment is submitted on Tuesday at 3:00 am, 8:00 am, 1:00 pm, 6:00 pm, and 11:00 pm. On Wednesday, the schedule starts over and the next submission is at 3:00 am.

Monthly Event

The map/reduce script deployment is submitted at least one time per month.

Yearly Event

The scheduled script deployment is submitted at least one time per year.

Start Date

The first submission occurs on this date. This field is required if a one-time or recurring schedule is set.

Start Time

If a value is selected, the first submission occurs at the time specified.

Repeat

If a value is selected, the first submission occurs on the date and time selected. A new script deployment instance is then created and submitted every x minutes or hours until the end of the start date. If applicable, the schedule starts over on the next scheduled day.

End By

If a value is entered, the last submission ends by this date. If you schedule the submission to recur every x minutes or hours, a new script deployment instance is created and submitted every x minutes or hours until the end date.

No End Date

The schedule does not have a set end date.

Body Fields

Status

Log Level

Priority

Concurrency Limit

Submit All Stages At Once

Yield After Minutes

Buffer Size

Schedule Subtab

Related Topics