Request Transformation Function

A request transformation (or transform) function is generally specified on the service endpoint. It can also be specified on the ServiceDataProvider variable, which overrides the endpoint one.

A request transform function is called right before a request is made to the server/endpoint. It provides a chance for page authors to transform the options (paginate, filter, sort, and so on) and build the final (request) configuration. The ServiceDataProvider supports a predefined list of request transform function types, described in this section. Note that there are no guarantees to the order in which transform functions are called.

A request transformation function has the following signature: function (configuration, options) { return configuration }, which is typically specified on the service endpoint. But it is also possible to specify this on the SDP variable, which overrides the one on the endpoint. The parameters to the function are:

• configuration: an object that has the following properties:

  • url: Full URL of the request.

  • readOnlyParameters: Path and query parameters. These are not writable.

  • initConfig: Map of another configuration passed into the request. The 'initConfig' exactly matches the 'init' parameter of the request.

• options: An object that is relevant to the type of transformation function. For a paginate function this would be the pagingCriteria.

• context: A context object that is passed to every transform function to store or retrieve any contextual information for the current request lifecycle.

If transformations are needed for a specific data provider instance, these functions can be defined on the ServiceDataProvider variable under the 'transforms' property. For externalized fetch cases, the RestAction properties can be used for configuring transformations. 

Types of Request Transform Functions

paginate

The  'pagingCriteria' is passed in as the 'options' parameter to the paginate function. The pagingCriteria is often based on the current paging/scrolled state of the component. 

• For implicit fetches, the pagingCriteria provided to the 'paginate' transform function can be used to construct a URL with the right paging query.

• For externalized fetches, the pagingCriteria is always set on the REST instance through the hook handler. This means that if the RestAction has a responseTransformFunctions.paginate transform function property configured, then it can expect the pagingCriteria to be provided to it.

For offset based paging:

•  size: Specifies how many items should be returned.

• offset: Specifies which item the response should begin from.

• The default value for the pagingCriteria can be set on the configuration, but generally a component that is bound to the ServiceDataProvider variable will provide the values, where offset and size will be based on the configuration set in the component.

  // Variable Configuration
  "incidentListTableSource": {
    "type": "vb/ServiceDataProvider2",     // variable of type vb/ServiceDataProvider
    "input": "none",
    "defaultValue": {
      "pagingCriteria": {                 // default size
        "size": 10
      },
   
      "transforms": {                     // transform function for paginate
        "request": {
          "paginate": "{{ $page.functions.paginate }}"
        }
      }
  }

  // paginate Transform Function
  // Transform function appends limit and offset parameters to the URL
  PageModule.prototype.paginate = function (configuration, options, context) { 
    const c = configuration;
    let newUrl = c.url;
    newUrl = `${newUrl}&limit=${options.size}&offset=${options.offset}`;
    c.url = newUrl;
    return c;
  };

filter

For this transform function, the 'filterCriterion' property is passed in as the 'options' parameter. The filterCriterion JSON property is an object representing a attribute criterion or a compound criterion. This example defines a simple structure for filter criteria that is a single criterion: 

// Variable Configuration
"incidentListTableSource": {
  "type": "vb/ServiceDataProvider2",
  "constructorParams": [
    {
      "transforms": {
        "request": {
          "filter": "{{ $page.functions.filter }}"  // transform function for filter; filterCriterion can be set in a list data provider view variable
        }
      }
    ]
  }
}

Here's a sample filter transform function that converts the filterCriterion property to a query parameter appropriate to the endpoint: 

/**
* Filter Transform Function Implementation 
* @param configuration - a Map containing the following properties 
* - url: url for the fetch 
* - readOnlyParameters: parameters that are appended to url 
* - endpointDefinition: endpoint metadata 
* - initConfig: an empty object 
* - fetchConfiguration: the configuration that triggered this fetch call. 
*   If fetch was initiated by SDP this includes fetch capability,
*   context, externalContext and fetchParameters property. Refer to the docs for the      
*   mergeTransformsOptions func callback for details. 
* @param options the JSON payload that defines the filterCriterion 
* @param context an object to store/retrieve any contextual information for the current request lifecycle. 
* @returns {object} configuration object. the url looks like ?filter=foo eq 'bar' */
 
PageModule.prototype.filter = function (configuration, options, context) {
  const c = configuration;
  const filterCriterion = options;
 
  function jetFilterOpToScim(fop) {
    switch (fop) {
      case '$eq':
        return 'eq';
      case '$ne':
        return 'ne';
      case '$co':
        return 'co';
      default:
        console.warn('unable to interpret the op ' + fop);
        return null;
    }
  }
 
  function isEmpty(val) {
    return (val === undefined || val === null || val === '');
  }
 
  if (typeof filterCriterion === 'object' && Object.keys(filterCriterion).length > 0) {
    if (filterCriterion.op && filterCriterion.attribute && 
          !isEmpty(filterCriterion.value)) {
      const atomicExpr = {};
      atomicExpr.op = jetFilterOpToScim(filterCriterion.op);
      atomicExpr.attribute = filterCriterion.attribute;
      atomicExpr.value = filterCriterion.value;
 
      if (atomicExpr.op && atomicExpr.attribute) {
        c.url = URI(c.url).addQuery({
          filter: `${atomicExpr.attribute} ${atomicExpr.op} ${atomicExpr.value}`,
        }).toString();
      }
    }
  }
 
  return c;
};

Write a Filter Transforms Function for Text Filtering

Text filtering is enabled by default.

If your SDP binds to a Business object REST API endpoint, you have the following options to get text filtering to work.

• Option 1: configure the SDP to include a vb-textFilterAttributes property where the attributes to apply the text filter is specified. The built-in Business object REST API transforms provided by Visual Builder looks for this property and automatically builds a filter criterion using the text and turns it into a 'q' param.

  • "transformsContext": {
    "vb-textFilterAttributes": ["lastName"]
    }

  • Example: if a user enters text 'foo' in select-single, for the configuration above SDP generates q=lastName LIKE 'foo%'
  • By default the operator used is 'startsWith' as this is considered to be more optimized for database queries that 'contains'
• Option 2: if the above doesn't meet your needs, then you can write a custom filter transform that massages the text filter and turns it into a regular filterCriterion.

If you use option 2 above, you could do something similar to the following example. In this example, resourcesListSDP uses the getall_resources endpoint. The (request) filter transforms property is a callback that is defined in the PageModule.

"resourcesListSDP": {
  "type": "vb/ServiceDataProvider2",
  "constructorParams": [{
    "endpoint": "crmRestApi11_12_0_0/getall_resources",
    "keyAttributes": "PartyNumber",
    "itemsPath": "items",
    "responseType": "page:getallResourcesResponse",
    "transformsContext": {
      "vb-textFilterAttributes": ["PartyName"]
    },
    "transforms": {
      "request": {
        "filter": "{{ $functions.processFilter }}"
      }
    }
  }]
}

It's important to note that the transformsContext object is an argument to every transforms function, so transforms authors can read the attributes and build the query that way.

The transforms function below takes the text value provided by the component and turns into an attribute filter criterion using the attributes passed in.

define(['vb/BusinessObjectsTransforms'], function(BOTransforms) {
  'use strict';

  var PageModule = function PageModule() {};

   /**
   * The filter transform parses the text filter that may be part of the options and replaces
   * it with an appropriate attribute filter criterion using the textFilterAttrs.
   *
   * Note: select-single provides a text filter in the form { text: 'someTextToSearch' }.
   *
   * The processing of the resulting filterCriterion is delegated to the Business Object REST API
   * transforms module, which takes the filterCriterion and turns it into the 'q' param.
   * @param textFilterAttrs
   * @return a transforms func that is called later with the options
   */
  PageModule.prototype.processFilter = function(config, options, transformsContext) {
    const c = configuration;
    let o = options;
    let textValue;
    let isCompound;
    const tc = transformsContext;
    const textFilterAttributes = tc && tc['vb-textFilterAttributes];
 
    textValue = o && o.text;
 
    // build your regular filtercriterion and delegate to VB BO REST API filter transforms
 
    return BOTransforms.request.filter(configuration, o);
  }
  return PageModule;
});

Note:

Page authors are discouraged from configuring the SDP with the 'q' parameter directly, for example by setting a 'q' parameter in the uriParameters property. It is recommended that authors always use filterCriterion property instead to define their 'q' criteria. This is especially important when using text filtering because the components always provide a filterCriterion which is appended to any configured filterCriterion on the SDP. It becomes especially difficult for Visual Builder to reconcile the 'q' defined in uriParameters with the filterCriterion and authors are on their own to merge the two.

It's also important to note that select-single calls fetchByKeys very often to locate the record(s) pertaining to the select keys. For this reason a new fetchByKeys transforms function has been added. Refer to fetchByKeys under transforms function for details.

sort

For this transform function, the 'sortCriteria' is passed in as the 'options' parameter. If page authors have complex sort expressions that cannot be expressed as a simple array, they can externalize the fetch to configure their own sort criteria and build a request using that.

// Variable Configuration
"incidentListTableSource": {
  "type": "vb/ServiceDataProvider2",
  "constructorParams": [
    {     
      "transforms": {
        "request": {
          "sort": "{{ $page.functions.sort }}"    // transform function for sort; sortCriteria can be defined in a listDataProviderView variable
        }
      }
    }
  ]
}

/**
 * Sort Transform Function Implementation
 * @param configuration - a Map containing the following properties 
 * - url: url for the fetch
 * - readOnlyParameters: parameters that are appended to url
 * - endpointDefinition: endpoint metadata
 * - initConfig: an empty object
 * - fetchConfiguration: the configuration that triggered this fetch call. If fetch was initiated by SDP this includes
 *   fetch capability, context, externalContext and fetchParameters property. Refer to the docs for the
 *   mergeTransformsOptions func callback for details.
 * @param options the JSON payload that defines the sortCriteria
 * @param context an object to store/retrieve any contextual information for the
 *  current request lifecycle.
 * @returns {object} configuration object. the url looks like ?orderBy=foo:asc
 */
PageModule.prototype.sort = function (configuration, options, context) {
  const c = configuration;
 
  if (options && Array.isArray(options) && options.length > 0) { 
    const firstItem = options[0]; 
    if (firstItem.name) { 
      const dir = firstItem.direction === 'descending' ? 'desc' : 'asc' 
      let newUrl = c.url; 
      newUrl = `${newUrl}&orderBy=${firstItem.attribute}:${dir}`; 
      c.url = newUrl; 
    } 
  } 
  return c; 
};

query

For this transform function, the 'uriParameters' property is passed in as options. Normally uriParameters are appended to the URL automatically, but there may be cases where the user would want to adjust the query parameters. For example, suppose the endpoint GET /incidents supports a query parameter called "search", which does a semantic search. If a specific transform needs to happen before the endpoint us cakked, then the transform function could be used for that. 

// Variable Configuration
"incidentListTableSource": {
  "type": "vb/ServiceDataProvider2",
  "constructorParams": [
    {
      "uriParameters":
        {
          "technician": "hcr",
          "search": "{{ $page.variables.searchBoxValue }}"// search query parameter 
                                                        // bound to some UI field
        }
      ],
     
      "transforms": {
        "request": {
          "query": "{{ $page.functions.query }}"       // transform function for query
        }
      }
    }
  ]
}

/**
 * Query Transform Function Implementation
 * @param configuration - a Map containing the following properties
 * - url: url for the fetch
 * - readOnlyParameters: parameters that are appended to url
 * - endpointDefinition: endpoint metadata
 * - initConfig: an empty object
 * - fetchConfiguration: the configuration that triggered this fetch call. If fetch was initiated by SDP this includes
 *   fetch capability, context, externalContext and fetchParameters property. Refer to the docs for the
 *   mergeTransformsOptions func callback for details.
 *
 * @param options the JSON payload that defines the uri parameters
 * @param context an object to store/retrieve any contextual information for the current request lifecycle.
 * @returns {object} configuration object. the url looks like ?orderBy=foo:asc 
 */
PageModule.prototype.query = function (configuration, options, context) {
  const c = configuration;
  if (options && options.search) {
    let newUrl = c.url;
      newUrl = `${newUrl}&search=${options.search} faq`; // appends 'faq' to the 
                                                         // search term 
      c.url = newUrl; 
	}
	return c;
  // configuration, options}; 
};

select

This transform typically uses the 'responseType' to construct a query parameter to select and expand the fields returned from the service. The built-in Business object REST API-based transforms (vb/BusinessObjectsTransforms) creates a 'fields' query parameter, such that the response will include all fields in the responseType structure, including expanded fields. For example:

/**
 * select transform function.
 * Example:
 *
 * Employee
 * - firstName
 * - lastName
 * - department
 *   - items[]
 *     - departmentName
 *     - location
 *        - items[]
 *          - locationName
 *
 * would result in this 'fields' query parameter:
 *
 * fields=firstName,lastName;department:departmentName;department.location:locationName
 *
 * @param configuration - a Map containing the following properties
 * - url: url for the fetch
 * - readOnlyParameters: parameters that are appended to url
 * - endpointDefinition: endpoint metadata
 * - initConfig: an empty object
 * - fetchConfiguration: the configuration that triggered this fetch call. If fetch was initiated by SDP this includes
 *   fetch capability, context, externalContext and fetchParameters property. Refer to the docs for the
 *   mergeTransformsOptions func callback for details.
 *
 * @param options
 * @param context a transforms context object that can be used by authors of transform
 *  functions to store contextual information for the duration of the request.
 */
PageModule.prototype.select = function(configuration, options, context) {
  // the options should contain a 'type' object, to override
  var c = configuration;

  // do nothing if it's not a GET
  if (c.endpointDefinition && c.endpointDefinition.method !== 'GET') {
    return c;
  }

  // do nothing if there's already a '?fields='
  if(queryParamExists(c.url, 'fields')) {
    return c;
  }

  // if there's an 'items', use its type; otherwise, use the whole type
  vartypeToInspect = (options && options.type && (options.type.items || options.type));
  if(typeToInspect && typeoftypeToInspect === 'object') {
    var fields; // just an example; query parameter construction is left to the 
                // developer
 
    if(fields) {
      c.url = appendToUrl(c.url, 'fields', fields);
    }
  }
  return c;
}

function appendToUrl(url, name, value) {
  // skip undefined and null
  if (value !== undefined && value !== null) {
    var sep = url.indexOf('?') >= 0 ? '&' : '?';
    return url + sep + name + '=' + value;
  }
  return url;
}
 
function queryParamExists(url, name) {
  const q = url.indexOf('?');
  if (q >= 0) {
    return (url.indexOf(`?${name}`) === q) || (url.indexOf(`&${name}`) > q);
  }
  return false;
}

body

The body request transform allows the author to modify the body payload for the fetch request. With some endpoints, the search is made with a complex search criteria set on the body that can be modified here.

The body transform function is called after all other transforms are run. This is to allow additional contextual information to be added by the previous transforms (to the transformsContext parameter) that need to be included in the body.

The transforms author can decide how to use the transformsContext property and the fetchMetadata parameter provided via the fetch call. The built-in Business object REST API transforms implementation currently does not use either.

/**
 * If a body is specified then we look for 'search' and 'technician' in the post body.
 * All other keys are ignored.
 * @param configuration - a Map containing the following properties
 * - url: url for the fetch
 * - readOnlyParameters: parameters that are appended to url
 * - endpointDefinition: endpoint metadata
 * - initConfig: an empty object
 * - fetchConfiguration: the configuration that triggered this fetch call. If fetch was initiated by SDP this includes
 *   fetch capability, context, externalContext and fetchParameters property. Refer to the docs for the
 *   mergeTransformsOptions func callback for details.
 *
 * @param options
 * @param context transforms context
 */
function bodyRequest(configuration, options, transformsContext) {
  const c = configuration;
  if (options && typeof options === 'object' && Object.keys(options).length > 0) {
    c.initConfig.body = c.initConfig.body || {};
    // update body
  }
  return c;
}

fetchByKeys

A fetchByKeys transforms function allows the page author to take a key or Set of keys passed in via the options and tweak the URL, to fetch the data for the requested keys.

When the consumer of the SDP calls the fetchByKeys() method, if the transforms author has provided a fetchByKeys transforms implementation then it gets called over the other transforms. If no fetchByKeys transforms function is provided then the default transforms will get called.

The built-in Business object REST API transforms already provides a fetchByKeys transforms function implementation that appends the keys to the URL. This should suffice for most common cases and should result in at most one fetch request to the server. For third-party REST endpoints, the author can provide a custom fetchByKeys transforms implementation.

In the following example for a sample third-party endpoint, the key is appended to the URL as a query param 'id=<key>'

define(['ojs/ojcore', 'urijs/URI'], function (oj, URI) {

  PageModule.prototype.fetchByKeysTransformsFunc = function (configuration, transformOptions){
    var c = configuration;
    var to = transformOptions || {};
    var fetchByKeys = !!(c && c.capability === 'fetchByKeys'); // this tells us that the current fetch call is a fetchByKeys

    if (fetchByKeys) {
      var keysArr = Array.from(c.fetchParameters.keys);
      var key = keysArr[0]; // grab the key provided by caller
      if (key) {
        c.url = URI(c.url).addQuery({
          id: key,
        }).toString();
      }
    }
    return c;
  };
});