Relevance Ranking Evaluator displays multiple search results side-by-side so you can compare and evaluate different test strategies.
Oracle recommends using an MDEX Engine that is in your staging or test environment so that the evaluation is not competing for resources in your production environment.
Relevance Ranking Evaluator can be used by more than one user simultaneously. Each user sees his or her own searches, and changes made by one user are not visible to others. Evaluations performed in the application are not stored outside of a user's browser session. If the browser is closed or if the session is allowed to time out due to inactivity, the evaluations are cleared and the application is reset to its initial state.
You should try a variety of common search terms in your evaluations. A relevance ranking strategy that seems to be perfect for one particular search term might not be appropriate for others, so it is important to evaluate several different types of searches.
To evaluate relevance ranking strategies:
Open Relevance Ranking Evaluator from Search Tools on the application Home page.
Enter a unique name for the strategy that you are evaluating in the Search Name field.
In the Search Criteria area, choose a search interface, match mode, and enter your search terms.
Each defined search allows for individual specification of the search criteria. While the most common use of Relevance Ranking Evaluator is to compare different relevance ranking strategies for the same search criteria, you can also use it to compare different combinations of search configuration options.
For instance, you can:
Ensure that spelling correction is behaving as expected, by comparing the results of searching for a correctly-entered term such as "camera" with the results of a search for a misspelled term such as "cammera."
Compare the behavior of different match modes, such as → and → .
Compare results from searching with slightly different search interfaces, such as one search interface that contains only short text properties like Name and Keywords, and another interface that contains the same short text properties and an additional longer Description field.
Select a Rollup Key to submit the search as an aggregated record navigation query. For more information on rollup keys, see the MDEX Engine Development Guide.
Click the plus sign next to Search Filters, and optionally configure a query-based filter.
Record Filters You can enter a record filter that limits the records returned and restricts the scope of the search results. For example, you can restrict searches to just those products that are in stock or when a data set contains multiple types of records like both products and reviews.
Here are some examples of record filters:
Range Filter You can apply a range filter to numeric attributes to restrict the records returned. Only one range filter can be applied.
Endeca Query Language (EQL) Filter You can apply an EQL filter to limit the records returned. An EQL filter can use multiple criteria, so it can be used in lieu of multiple range filters. Properties used in EQL must be enabled for record filters. In addition, the attribute names must be NC_NAME compliant: they must not contain spaces. The power of EQL is the ability to include complex Boolean logic across various filter types.
In the Relevance Ranking area, select the relevance ranking modules that you want to apply for the search. For descriptions of the modules, see Relevance ranking modules
A set of relevance ranking modules, combined with the order in which they are evaluated, is referred to as a relevance ranking strategy. You can select one of the predefined or stored strategies, or create your own custom strategy. Follow these steps:
Click a module in the Available Modules list.
A description of the module appears in the status bar at the bottom of the Relevance Ranking area.
Note
Some modules might require you to set additional configuration options. Set these options before you continue to the next step.
Click the right arrow to move the module to the Selected Modules list.
If you change your mind, click the left arrow to move the module out of the Selected Modules list.
Repeat the previous two steps until you have defined your relevance ranking strategy.
Strategies that contain four or five modules are unlikely to produce results that are significantly different from strategies containing fewer modules.
Click the up and down arrows next to the Selected Modules list to re-order your list of modules.
This order determines the precedence of the modules when the evaluator orders search results.
Select the number of records that you want to display in your results in the Show drop-down list at the top of the evaluator page.
If you want to create more strategies to compare your initial one with, click the plus (+) tab next to the Search tab at the top of the evaluator page.
Results for the current search strategy display, and a new search displays already populated with the values from the first strategy.
Note
Try not to click the plus (+) tab next to the Search tab until you have finished configuring each strategy because this action causes Evaluator to run and display results.
After you have finished configuring your strategies, click Go next to the Search Name field.
Each set of search results displays in its own column in the Search Results Comparison section at the bottom of the page. The header lists the name of the search and the number of records matched. To help you better understand the results, each search results header provides detailed information about the search that was performed. Click the plus sign to the right of the search name and results count to see this information.
The Relevance Ranking Score property does not represent a percentage score. It is an internal calculation that the MDEX Engine uses to determine ranking. A higher number equates to greater relevancy based on the applied strategy. To display additional attributes for each record, click Show All Properties And Dimensions. If you have deleted any attributes that display, you can redisplay them by clicking Hide All Properties And Dimensions, and then clicking Show All Properties And Dimensions again.
You can compare search results, including where a given record is ranked in each set. Place your mouse over the displayed properties for the record. When your mouse is placed over the record properties, the record is highlighted in any search in which it is displayed. A tooltip appears that provides the location of the record in each set of search results
