• Is the long processing time for the Engine caused by limitations of hardware resources? Identify whether long query time is caused by CPU, memory, or disk I/O utilization.

• Is a high number of records being returned by the MDEX Engine? Identify how many records are being returned per query by looking for large nbins values in queries as reported by the Request Log Analyzer. This value indicates the maximum number of records that can be returned in the query. If this number is high, this can be expensive to compute and affects performance. Consider implementing paging control methods. For information on using paging control methods, see the MDEX Engine Developer's Guide.

• Are all dimension refinements (dimension values) exposed for navigation? That is, examine whether your queries are spending most of their time in refinement computation. Identify whether all dimension refinements are exposed by looking for allgroups=1 in the Dgraph request log (request URL parameter) or in Request Log Analyzer reports.

  This setting corresponds to NavAllRefinements value of the ENEQuery method.

  If the allgroups=1 setting is present in the URL parameter, review this configuration setting for your application to decide whether it is necessary. Exposing all refinements for navigation can decrease performance because the MDEX Engine has to examine each dimension value in the dimensions and determine whether or not that dimension value is a valid refinement given a current navigation state. Exposing all dimension refinements for navigation is not recommended.

  For dimensions with many dimension values, Oracle recommends introducing a hierarchy (for example, a sift dimension hierarchy for automatically generated dimensions), so that the MDEX Engine has fewer dimension values to consider at one time.

• Are your longest queries similar? Check the longest queries for similarities, such as whether they all use the same search interface with relevance ranking, wildcard search, or record filters. See the sections in this guide about tuning performance of each of these features.

• Is record search being used? Identify whether a record search is being used by any queries by looking for “attrs=search_interface_name” in a query. This indicates that a record search is being used which means that possibly expensive relevance ranking modules can be contributing to high computation time.

• Which relevance ranking strategies are being used? Check the app_prefix.relrank_strategies.xml file for the presence of Exact, Phrase and Proximity ranking modules and test the same query with these modules removed.

• Is sorting enabled for properties or dimensions? Identify whether sorting with sort keys is enabled, for which properties and dimensions it is being used and whether it is needed. The first time a sort key is issued to a Dgraph after startup the key must be computed which can slow down performance. To isolate this problem, test the query in the staging environment by removing the sort key. If you confirm sort keys are the issue, consider using sort keys in a representative batch of queries used to warm up the Dgraph after startup. The sorts will become cached and these queries will be faster.

  Note

  Also, identify if sorting for properties and dimensions is necessary. In particular, it is not necessary to flag all sortable properties as sort keys in the project. This is often a performance problem itself.

Related links

• CPU recommendations for optimizing performance

• I/O recommendations for optimizing performance

• Disk access recommendations for optimizing performance

• Relevance ranking

• Disk I/O problems can sometimes cause slow warming.

• It is helpful to run a Dgraph warming script at startup. For example, you can use a request log of characteristic queries played against the Dgraph to help warm it to a steady state.

• You are using too many threads in one process. This is unlikely unless you exceed four threads, in which case consider using multiple Dgraphs.

• You have an I/O problem.

• There is an underlying network problem that needs to be investigated.

• In most cases, the following recommendation applies: Dgraphs with a large memory footprint, especially in search-intensive applications, should be run in multithreaded mode with the number of threads greater than one for best performance.

  For example, suppose you have a four-processor 16GB machine and a 3GB Dgraph. You could run four identical separate Dgraphs. A better alternative is to run one four-threaded Dgraph and thus reap the benefits of having more disk cache.

  By running with more than one thread, I/O and computation can be overlapped. Although the time to process an individual request isn’t improved (and can actually increase slightly due to contention for shared resources), overall throughput is significantly boosted.

• Likewise, in many cases it is appropriate to run two or more Dgraphs on one machine, each with several threads. Two four-threaded Dgraphs on one machine is an especially common configuration. The trade-off between thread contention and memory depends on the memory footprint that you estimate is needed for each Dgraph and the amount of memory available on the machine that will host multiple Dgraphs.

• Use a dedicated storage device with low latency and high IO ops/sec for all your indexes and files. Locally-attached storage with a RAID controller is preferred. Only in cases where that is not possible, SAN using a Fibre Channel will typically provide strong performance assuming it has been configured correctly.

• If you are using an array controller, Oracle recommends using a striped disk configuration, such as RAID 5/6 or RAID 0+1 that enable you to avoid having redundant disks but ensures fault tolerance.

• Do not use disks with NFS, or other file system protocols. They are known to slow down performance.

• Ensure that the log files are saved locally. Turning off verbose mode, which prints information about each request to stdout, can sometimes help performance.

• Ensure that you have a fast disk subsystem and plenty of memory available for disk cache managed by the operating system, since the Dgraph keeps its various text search indices on disk, including search and navigation indexes.

• If the CPU is under-utilized, increase the number of threads for the Dgraph.

• If the CPU is over-utilized and you are not satisfied with throughput, investigate which activities make it busy. Add machines or make the queries less taxing by tuning individual features.

Related links

• Dgraph Analysis and Tuning

• On Solaris, run iostat -2

• On Linux, run sar -b

• On Windows, do the following:

  On the Task Manager, open the Processes tab.

  From the menus, select View → Select Columns.

  Check I/O Reads, I/O Read Bytes, I/O Writes, and I/O Write Bytes. These options enable new columns in the Processes pane that provide similar information to sar -b on UNIX.