Feasibility Surveys

This section describes how survey managers create, choose, and score feasibility surveys using the Alchemer tool and Oracle Site Select. Surveys you create in Alchemer will be configured in Oracle Site Select and communicated to sites via workflow task. You'll also learn about the total survey points and where you can see this information in Oracle Site Select.

Alchemer online survey tool and supported question types

If you have the Feasibility Survey Manager permission, you'll use Alchemer online survey tools to create and manage feasibility surveys in Oracle Site Select.

After creating a survey in using the Alchemer product, you can then associate that survey to a study and configure the survey name, flag it as the default, apply scoring and ranking as preferred from within Oracle Site Select. On a survey's Definition tab, you'll find a "View your Alchemer account surveys" link that will open the Alchemer login page in a new browser tab.

Tip:

When creating an Alchemer survey, use sections to add and organize related content if another section heading is needed.

Alchemer offers many question types that allow your organization to create highly customized feasibility surveys. The following tables detail question types supported in Oracle Site Select and those that are not supported. The table also includes details about options and scenarios we've tested.

Question Type	Logic Tested with Oracle Site Select	Supported by Oracle Site Select	Scorable in Oracle Site Select	Tested As	Additional Conditions	Supported in Oracle Site Analyze
Textbox	Skip, Display	Yes	No	N/A	None noted in testing	Yes
Textbox List	This type does not allow additional logic	Yes	Yes	2, 4, 6, 8, 10	None noted in testing	No
Textbox Grid	This type does not allow additional logic	Yes	No	2x2, 4x4, 6x6, 8x8, 10x10, 20x20	None noted in testing	No
Slider	Skip, Display	Yes	No	N/A	None noted in testing	Yes
Rating (Likert Scale)	Skip, Display	Yes	Yes	6, 10, 20	None noted in testing	Yes
Radio Button Grid	Skip, Display	Yes	No	2x2, 10x10, 20x20	None noted in testing	No
Radio Button	Skip, Display	Yes	Yes See Additional Conditions	4x4, 10x10, 20x20	Scorable without customization. Using the custom response option, the data does NOT come into Oracle Site Select and therefore is not scored.	Yes
Percent	Skip, Display	Yes	Yes	N/A	None noted in testing	Yes
Number	Skip, Display	Yes	Yes	N/A	None noted in testing	Yes
Net Promoter Score (NPS)	Skip, Display	Yes	Yes	N/A	None noted in testing	Yes
Image Select (select one)	Skip, Display	Yes	Yes	3, 4, 5	Advanced logic not supported. Known behavior: Oracle Site Select PDF shows both "no answer" and list of image file names.	Yes
Image Select (multi- select)	Skip, Display	Yes	Yes	4, 7	Advanced logic not supported. Image file name included in PDF download and CSV export	No
Essay/ Long Answer	Skip, Display	Yes	No	N/A	None noted in testing	Yes
Email	Skip, Display	Yes	No	N/A	None noted in testing	Yes
Drop-down menu	Skip, Display	Yes	Yes	N/A	None noted in testing	Yes
Drag and Drop Ranking	This type does not allow additional logic	Yes	No	8x8, 20x20	None noted in testing	No
Date	Skip, Display	Yes	No	N/A	None noted in testing	Yes
Continuous Sum	This type does not allow additional logic	Yes	Yes	2x2, 6x6, 20x20	None noted in testing	No
Checkboxes	Skip, Display	Yes	Yes	4, 8, 10, 20	Oracle Site Select does not support the checkbox logic option "Other."	Yes
Checkbox Grid	Skip, Display	Yes	Yes	8x8, 10x10, 20x20	Known behavior: Does not work for reversioning of surveys; 8x8 option shows incorrect order in UI/PDF	No
Contact Form	Skip, Display	Yes	No	N/A	None noted in testing	No
Signature	This type does not allow additional logic	No	No	N/A	None noted in testing	No
File Upload	This type does not allow additional logic	No	No	N/A	None noted in testing	No
Star Rating Grid	Skip, Display	Yes	No	10x10, 20x20	Limited to 10x10 option. 20x20 can cause unexpected behavior.	No
Ranking Grid	This type does not allow additional logic	Yes	No	10x10, 20x20	None noted in testing	No
Slider List	This type does not allow additional logic	Yes	No	10x10, 20x20	Limited to 10x10 option. 20x20 can cause unexpected behavior.	No
Drop-down Menu List	Skip, Display	Yes	No	6x6, 10x10, 20x20	None noted in testing	No
Drop-down Menu Grid	Skip, Display	Yes	No	10x10, 20x20	Limited to 10x10 option. 20x20 can cause unexpected behavior.	No

The following table defines export for Alchemer question types listed above:

Question Type	Export from Select via PDF	Export from Select via CSV	Export from Analyze via CSV
Textboxes	Yes	Yes	Yes
Textbox List	Yes. Each option listed with input.	Yes. Each row exports as a column.	No
Textbox Grid	Yes	Yes. Each grid row exports as a separate column.	No
Slider	Yes	Yes	Yes
Rating (Likert Scale)	Yes	Yes	Yes
Radio Button Grid	Yes	Yes. Each grid row exports as a separate column.	No
Radio Button	Yes	Yes	Yes
Percent	Yes	Yes	Yes
Number	Yes	Yes	Yes
Net Promotor Score (NPS)	Yes	Yes	Yes
Image Select (select one)	Yes	Yes	Yes
Image Select (multi-select)	Yes - selected file name	Yes - selected file name	Yes
Essay/Long Answer	Yes	Yes	Yes
Email	Yes	Yes	Yes
Dropdown Menu	Yes	Yes	Yes
Drag & Drop Ranking	Yes. Answers appended with -add.	Yes. Each grid row exports as a separate column.	No
Date	Yes	Yes	Yes
Continuous Sum	Yes. Shows options selected but not total value.	Yes. Each selected option exports as a column.	No
Checkboxes	Yes	Yes	Yes
Checkbox Grid	Yes	Yes. Each grid row exports as a separate column.	No
Contact Form	Yes	Yes. Each field exports as a separate column.	No
Signature	No	No	No
File Upload	No	No	No
Star Rating Grid	Yes	Yes. Each grid row exports as a separate column.	No
Ranking Grid	No	Yes. Each grid row exports as a separate column.	No
Slider List	Yes	Yes. Each grid row exports as a separate column.	No
Dropdown Menu List	Yes	Yes. Each row exports as a separate column.	No
Dropdown Menu Grid	Yes	Yes. Each grid row exports as a separate column.	No

The following Alchemer survey question types are not supported.

Question Type	Comments
Video Sentiment	This type requires additional cost for a higher license level and will not be supported.
Audio Sentiment	This type requires additional cost for a higher license level and will not be supported.
Text Highlighter	This type requires additional cost for a higher license level and will not be supported.
User Data	This is data collected only on Alchemer users and will not be supported.
Cascading Drop-down Menu	This type requires additional cost for a higher license level and will not be supported.
Conjoint (Choice-Based)	This type requires additional cost for a higher license level and will not be supported.
Semantic Diff	This type requires additional cost for a higher license level and will not be supported.
Quick Sort	This type requires additional cost for a higher license level and will not be supported.
Max Diff	This type requires additional cost for a higher license level and will not be supported.
Image Heatmap	This type requires additional cost for a higher license level and will not be supported.
Grouping (Open Card Slot)	This type requires additional cost for a higher license level and will not be supported.
Grouping (Closed Card Slot)	This type requires additional cost for a higher license level and will not be supported.
Custom Table	This type requires multiple levels of logic and will not be supported.
Custom Group	This type requires multiple levels of logic and will not be supported.

Feasibility survey configuration

When you have the Oracle Site Select Feasibility Survey Management role permission, you can configure and manage your study's surveys on the Configure study feasibility surveys page. The page lists the study's saved surveys and includes distinguishing details on each survey row to provide more information at a glance. For instance, the list shows:

• Survey short name
• Survey Alchemer name
• Workflows: (assigned workflows list)
• Last updated: <date in DDMonYYYY>; Created: <date in DDMonYYYY>

The list also shows question count details for each listed survey. Question counts are clickable links to the settings area for that detail line (e.g., the questions/scored/critical preview numbers on the survey list link to the scoring detail subtab for that survey). The links are:

• # questions
• # scored questions
• # critical questions (design may show unscored questions)
• Ranking enabled (displays only when enabled)

A badge displays on the far right of the survey row if a survey has a specific type (i.e., Default or Preferred site). Additionally, status badges show you whether the survey is Available, In setup, Sent to sites, or Not used.

The configuration list page provides additional functionality. For example:

• The search field above the studies list allows you to search for a specific survey by the survey short name or Alchemer name. This field is a “fuzzy” search that returns close matches and exact matches.
• Category, tiering, and export settings for surveys apply at the study level, and you can access these settings from the configuration page. Above the study list, click the Configure categories, Configure tiering, or Export scoring config buttons to open a settings drawer from the right side of your screen. Each of these configuration options open a separate settings drawer.
  • Category: Use the settings in the Define categories drawer to add new, unique categories (25 character max) and delete categories, if necessary. For an existing study with survey categories already defined, the previously-saved values will display in the category config drawer. If you attempt to delete a category that's already assigned to a question, an error displays and lists the surveys where the category is already used so you can assign a new category to the affected questions before attempting to delete again.
  • Tiering: Use the settings in the Define scoring tiers drawer to define ranges for each of the four survey tiers. For an existing study with existing tiering configured (other than the default) the previously-saved values will display on the tiering config drawer.
  • Export scoring: Use the settings in the Export survey scoring drawer to your preferences for the Microsoft Excel format export. You can choose your report format (either Questions as rows, Questions as columns, or Enhanced questions and criteria report) and your preferred reporting options (Only export questions marked scored and/or Export questions comments).
• If none of the surveys listed on the Configure study feasibility surveys page meet your study requirements, click the Add new survey button to navigate to the survey definition tab, where you can choose a survey from the list of the surveys configured in your connected Alchemer account and configure it as required for the study.

Preferred site surveys

When there is at least one preferred site in any datasource attached to the study, Oracle Site Select automatically adds a Preferred site survey to the survey list on the Configure study feasibility surveys page. The survey will have a white "Preferred site" badge. By design, Oracle Site Select won’t add a Preferred survey if there isn’t a preferred site in an attached datasource, and you may not create a survey with “Preferred site” as the short name.

On the Preferred site survey’s Definition tab, below the Study survey short name field (which is not editable), you’ll see a check box option with the following label and helper text: “Disable this survey option so preferred sites receive the survey they are assigned in their workflow. Note: If this option is disabled you must select a survey below and save changes. All preferred sites in the study will receive this survey regardless of what survey is assigned in their study workflow. This may cause confusion with your study team.” By default, the check box is checked.

If there is a Preferred site survey and you configure it to be available for all preferred sites (i.e., the checkbox is unselected for “Disable this survey option….”), choose a survey and configure the survey specific settings on the Scoring and Ranking tabs. You can also create a Survey submission message for the Preferred site survey.

Survey definition settings

To adjust settings for an individual survey listed on the Configure study feasibility surveys page, click the arrow at the far right of the survey row to access the Configure feasibility survey, Definition tab. Here, you'll find settings for selecting a survey for the study and configuring it to your needs. Functionality like setting the survey's short name, marking it as the default, creating a survey submission message, etc. is also available on this page. Note that if you choose to update the submission message after already assigning the survey to sites, you'll need to click Update message to ensure the new message will be used going forward. Click Edit survey at the top left of the Definition tab to sign in to Alchemer and modify the selected short name survey, if necessary. Click Preview survey to view the survey in a new browser tab.

Click the Scoring tab to define the survey's question scoring. When defining scoring settings, you can update the Category and Scored? column values, in line, by clicking the applicable table cell (look for the down arrow that opens a list of values). If you need to update other values for the question, click edit to open the question's scoring drawer component. After making changes, click Done to close the drawer, and be sure to click Save on the Configure feasibility survey page to ensure your changes take effect.

Note:

We recently made a change to scoring configuration to allow for saving and calculating a survey score without categorizing the scorable questions. With this change, the survey's Survey total score is based on the total points achieved from responses to scored scoreable questions divided by the maximum points available for each scored scoreable question in the survey.

You can optionally award score values to survey question responses based on a scale determined by how the response compares against all other inputted responses. These “Floating scale” scores apply only to textbox input questions with numeric and percentage inputs. When you choose Floating scale scoring, the drawer displays four ranking inputs, each having its own score input. If the question isn’t marked as scored, the ranking input configuration is disabled, but it will persist any inputted criteria. Please note that marking a range as critical is not supported for Floating scale scoring.

When you set up Floating scale scoring:

• For textbox list questions and continuous sum questions, each row of the question is considered a separate textbox input question. For example, a textbox list question with three rows is considered three separate textbox input questions, and you can apply a floating score to each.
• For textbox grid questions (with numeric or percentage validation), each row:column pairing is considered a separate textbox input question.

The site grid Survey analysis tab has a Rescore button that is enabled when any survey in the study has a question configured for float scoring. When clicked, Oracle Site Select ranks survey responses to the question marked for a floating scale and applies the point value corresponding to the percentile ranking for the site’s question response. Each time a Oracle Site Select site grid user clicks the Rescore button, Floating scale scoring recalculates across every survey in the study.

IMPORTANT: Oracle Site Select considers the latest responses to a survey with floating point questions when scoring the survey across all sites that have submitted it. If a study site has been reassigned to a new survey, the previous latest response submission of the survey with a floating point question will still be considered in the scoring of all surveys with that floating point question. This is regardless of the fact that the study site is no longer assigned to that survey with the floating point question.

On the Configure feasibility survey page, click the Ranking tab to configure survey ranking to your preferences within the drawer component. Survey ranking is an optional feature that groups sites based on their performance against their peers. For example, when 10 site surveys are submitted, and Rank 1 is defined as the top 80% of sites, the 2 sites with the highest scores will receive a rank of 1. When survey ranking is configured, site grid users can add a Survey ranking column to the grid and the column can be included in data extracts, reports, etc. Chapter 12 Scoring surveys 12When Ranking has been enabled for the survey, you'll see a green check mark next to the Ranking tab.

Survey started at or by

Oracle Site Select records the survey as started when either an Oracle Site Select user or a Site portal user clicks the survey link for a particular site.

Survey started at/by values will be recorded as follows:

• When a Site portal user reaches the survey task, the survey is considered started, and the started at/by values are set based on the user who clicks the link to the survey from the Site portal survey task action.
• If an Oracle Site Select user completes the previous task, they are recorded as the user who ungated or unblocked the survey task. If the workflow’s first task is the Survey task, the first user to access the Site portal and the time they first accessed it are recorded as the started at/by values
• If the survey started at/by values are not already populated and a Select user opens the survey via the site Workflow tab in the Site details drawer, that user will be recorded as the survey started at/by value.
• If a site starts a survey in one workflow (e.g., SurveyA), is reassigned to another workflow with a different survey (e.g., SurveyB), and then returns to the previous workflow (SurveyA), the started by/at values for SurveyA will be restored.
• Editing a survey does not update or change the started by/at values.

Total survey points

Total survey points improves visibility into feasibility survey scoring, and there are several options for you to view this important information. Using the column picker, you can optionally add the Total Survey Points column to see the total in the context of the site grid. The column displays the total possible points a site could have received on its most recent survey submission, taking into account skip logic, branching paths, and question type (single, multiple, or grid).

The score shown in the Survey score area of the survey response modal also shows Total survey points. You can see both the raw score and the possible score for all submitted survey versions. If a survey was submitted prior to Oracle Site Select release 25.1, resave the scoring configuration page to view the possible points. Resaving triggers recalculation and population of a new scoring field for that survey.

Survey percentage score and category-level scoring logic also use the actual possible points attained during survey completion. This ensures the percentage reflects only the questions that were shown and answerable by the site.

We created a total_possible_score column to the survey_response table to support this functionality. The name of the column aligns with the current column names and differentiates possible points from the existing "raw_score" column.

The total_possible_score column is also available in nightly CSVs files for reporting views in the study_sites and survey_responses, and as follows in REST API endpoints:

• workflow_details - added to the survey_response_details object array: For each submitted survey, will populate the total possible points calculated as "survey_total_possible_score" and will populate the total points earned by the site for the survey as "survey_total_points_scored"
• studysite_details - this api is a dump of the studysite table. It outputs only the most recent survey submission version possible points as "survey_possible_score"

• Set up feasibility survey
  Configure one or more feasibility surveys for use with a study. When a study has multiple associated surveys, at the workflow level, users can determine which survey to affiliate with which workflow.
• Set up tiering
  Survey tiering is based on the survey response total percentage score. The ranges can be updated as needed and tiers will be recalculated immediately for all survey submissions upon saving changes. Survey tiering is configured at the study level.
• Export survey scoring configuration
  If you have permission to set up feasibility surveys, you can optionally export a study's survey scoring configuration to a Microsoft XLSX file.
• Edit existing surveys
  
• Preview a survey
  Preview a survey to confirm expected layout and question formatting.