orch.pack

Compresses one or more in-memory R objects that the mappers or reducers must write as the values in key-value pairs.

Usage

orch.pack(...)

Arguments

. . .: One or more variables, data frames, or other in-memory objects in a comma-separated list.

Usage Notes

You should use this function when passing nonscalar or complex R objects, such as data frames and R classes, between the mapper and reducer functions. You do not need to use it on scalar or other simple objects. You can use orch.pack to vary the data formats, data sets, and variable names for each output value.

You should also use orch.pack when storing the resultant data set in HDFS. The compressed data set is not corrupted by being stored in an HDFS file.

The orch.pack function must always be followed by the orch.unpack function to restore the data to a usable format.

Return Value

Compressed character-type data as a long string with no special characters

Example

This code fragment compresses the content of several R objects into a serialized stream using orch.pack, and then creates key-value pairs using orch.keyval:

orch.keyval(NULL, orch.pack(
     r = r,
     qy = qy,
     yy = yy,
     nRows = nRows))