Tips for optimizing XQuery

This topic contains tips for optimizing XQuery.

Note: Several of the examples in this topic are shown in the format used by the command-line developer tool, exquery, that is provided by Endeca. You can use exquery in interactive or file-based mode to test and debug the XQuery functions that you write yourself. More information about exquery appears in the chapter "Getting Started with Web Services and XQuery for Endeca."

Use the ep:time pragma as a profiling tool to find hotspots. Only optimize code that is expensive to execute. However, keep in mind that the ep:time pragma may produce unintuitive results because of XQuery optimization. For example, consider the following code:
```
declare namespace ep = "http://www.endeca.com/XQuery/pragmas/2008";
 (#ep:time all#) {
 let $n := (#ep:time seqstuff#) { reverse(1 to 1000000)[1] }
    return (#ep:time add#) { $n + 1 }
 }
```
which, run in exquery, emits the following:
```
TRACE{seqstuff}: 821.809 ms
 TRACE{add}: 822.199 ms
 TRACE{all}: 822.229 ms
 1000001
```
Not only does this imply that the all block take less time than the sum of its children, but also that creating and reversing a million-element list takes as long as performing addition. The cost of addition is actually trivial, but the XQuery compiler performed let-inlining to produce code structured like this:
```
declare namespace ep = "http://www.endeca.com/XQuery/pragmas/2008";
 (#ep:time all#) {
    return (#ep:time add#) { (#ep:time seqstuff#) { reverse(1 to 1000000)[1] } + 1 }  }
```
To investigate cases where the optimizer may have interfered with the time pragma, you can put the time pragma around larger blocks of code to look out for inconsistencies, or use the !o modifier in exquery to see what the optimizer does to various kinds of expressions.

Do not calculate the same value, or walk down the same path, repeatedly in performance-critical code. For example, instead of:

<packed>
  <fish>{$x/categories/foodstuffs/[@type = 'fish']}</fish>
  <bread>{$x/categories/foodstuffs/[@type = 'bread']}</bread> </packed>

use:

  let $foodstuffs := $x/categories/foodstuffs return
  <packed>
    <fish>{$foodstuffs/[@type = 'fish']}</fish>
    <bread>{$foodstuffs/[@type = 'bread']}</bread>
  </packed>

This can save a significant amount of time, if $x or $x/categories or $x/categories/foodstuffs has many children. (Common sub-expression elimination has not yet been implemented.)

The XQuery evaluator is not optimized to work on large results sets. If you bring a large data set into XQuery, it will have to be internally materialized and processed, even if XQuery does not emit a lot of data. Do as much work like filtering and searching as possible in the queries executed by the Dgraph, rather than in XQuery code.
Because sorting is expensive, you should not use the order by clause in FLOWR expressions more than is necessary. In addition, you should minimize the size of the sequence values that are sorted.
The Endeca XQuery evaluator does not perform tail-call optimization or limit recursion depth. Therefore, recursive functions may execute slowly, and deep recursion may cause a stack overflow.