Test Builder

Test Builder Overview

The Test Builder enables you to create and manage tests either as topics within a course, or as courses in their own right. A test is a means of measuring a learner's performance, knowledge, and skills. Tests can be scored or not scored. For example, tests are scored and evaluations are not scored. When a test is scored, and a mastery score is specified, a learner passes or fails a test based on the results. Tests that are not scored can provide survey functionality.

Note: The Test Builder is an integral part of the application, and is available to all administrative users, assuming the Online Learning option is enabled for the application. No licence or separate software is required.

The test building process follows a cycle of tasks as illustrated below:

The Test Building Process

Key Concepts

To use the Test Builder effectively, you must understand the following key concepts:

• Tasks to Assemble a Test

  Before attempting to create and offer a test, first understand the tasks involved.

• Question Banks

  A question bank is a repository of questions and responses. Question banks help you to manage and reuse questions in multiple tests. At least one question bank must exist before you create a test.

• Test Creation

  After you have a created question bank and added the required questions, you can create a test and define the test properties.

• Section Management

  Tests comprise test sections, that contain questions drawn from one or more question banks. You create test sections to organize the questions drawn from a question bank. A test must contain at least one test section and at least one question.

• Test Preview and Delivery

  You can preview questions at different stages of test assembly. Delivering the test is the final step in the test building process. You can deliver a test either as a topic within a course or as a separate course.

• Reviewing Test and Question Statistics

  The Test Builder provides a number of statistical screens for you to analyze learners' performance and test results, either for the test or for individual questions.

• Viewing Free Text Responses

  If a test for a class includes free text questions, then you can view the responses using the Free Text Responses link on the Content tab for a test. You can search for a class and then view learners' response to the free text questions.

  When you search for a class, OLM displays:

  • Classes to which an evaluation is added at the class level.

  • Classes to which a test or survey is attached at the offering level.

  On the search results table, you can view information such as the learner name, class name, question text, and date and time of the response. You can click the View Response icon to view the question and response details.

  Note: The Learner column displays the learner's name if the free text response is a part of test or survey. OLM does not display the learner's name if the free text response is part of an evaluation. To hide the learner's identity, by default OLM displays the asterisk instead of the learner's name. If required, the system administrator in your enterprise can display the names using the Personalization function.

Test Builder Requirements

Can you create more than one test that reuses the same questions?

Yes, you can create many questions in a question bank, for example all related questions for a specific subject, and reuse those same questions in many tests.

Can you import tests created by a third party content provider?

Yes, if the tests conform to the IMS QTI (Question and Test Interoperability) specification. Use the Import function at either the folder or the test level to import tests.

See: Importing and Exporting Tests

What question types can you create?

The Test Builder supports the following question types: Fill-in-the-Blank (text response), Fill-in-the-Blank (numeric response), True/False, Multiple Choice Single Response, Multiple Choice Multiple Response, Multiple Choice Multiple Response, and Free Text (Not Scored).

Can you display multimedia files and hyperlinks in test questions or responses?

Yes, all text fields support HTML code if you select the Render HTML option when creating or editing your test.

Can you delete questions from a question bank?

Yes, but only if the question has not yet been attempted. Once a learner has attempted the question in any test, you can no longer delete the question from the question bank, since it is required for learning history and performance records. If a question is no longer applicable for future tests, you can make it inactive, to prevent the administrator from selecting the question in any future test.

Can you test learners' knowledge at intervals throughout a course?

Yes, one or more tests can exist as part of a content structure, which is offered as an entire course, so that you test learners' knowledge as they progress through the course.

Alternatively, a test can be a separate course in its own right.

Can you force learners to take a test before launching specific online content?

Yes, you can define the test to be a player prerequisite for another online learning object, to prevent learners from attempting subsequent sections of a course, or a separate course, until they have successfully passed the test.

Can you limit the time a learner takes to complete a test?

Yes, you can specify a time limit for a test (a timed test), to restrict the amount of time a learner has to complete a test and to alert the learner as the time limit approaches.

Test Assembly

Test Assembly Tasks

The first task in creating tests is to ensure that the question banks and questions exist, and if not, to create them. Once the questions exist and the question bank is published, you can create a test by adding the questions required. When preparing to create a test, ask yourself the following:

• Do the questions required already exist? If not, create questions in existing or new question banks.

• Is the test a topic in a course or a standalone course? This placement determines where in the content hierarchy you create the new test object.

• Is the test a simple, one-section test that uses all, or the majority of, questions in a question bank? If so, create a quick test from a question bank, and modify the test properties. If not, create a test object explicitly.

• Do you want to provide random questions from a question bank each time a learner launches the test, rather than providing the same specified questions to all learners? If so, create the test yourself. You cannot use the quick test method, since this builds a test using all questions explicitly from the bank. The quick test method does not support the Rule-Based test option that dynamically extracts a given number of questions from a bank at run time.

Question Bank Creation

You create question banks within a folder. When a question bank is ready for use in a test, you publish the question bank. After publication, you can use the question bank to create tests, or select the Quick Test option to create a test based on all the questions in the bank.

Creating Questions

Question banks consist of questions, each containing the question text (or question stem or prompt), possible responses, and feedback (if any). The feedback typically informs the test taker if they have answered correctly, and can include a learning tip.

When you create a question, you decide what type it is, such as multiple choice, free text, or true-or-false. Once you create the question, you cannot change from one type to another, since the style of responses depends on the question type. The exception is for multiple choice, where you can change from a multiple choice single correct response to a multiple choice multiple correct response.

Creating Responses

A response is a possible answer to a question. Each question has a set of responses, depending on the question type:

• True or False: The two values, True and False, are not editable. Select the correct response value, either True or False.

• Fill-in-the-Blank: Enter one or more correct responses. Text responses are case-sensitive, so enter the correct response in both lower and upper case, if either is correct.

• Multiple Choice (Single Correct Response): Enter two or more response values, from which the learner must select one correct choice.

• Multiple Choice (Multiple Correct Responses): Enter three or more response values, from which the learner must select two or more correct choices. Learners must select all the correct responses to gain the points awarded to the question.

• Free Text (Not Scored): The free text question is not scored and does not have the correct response validation. You can provide feedback, if required. Instructors must manually check the responses of the free text questions to decide whether responses are satisfactory.

Removing Questions or Responses

Once you use a question in one or more tests, and a learner has taken that test, you cannot delete the question from the question bank. Similarly, you cannot delete a response value if the question already appears in a test and if at least one learner has answered that question. You can mark a question or response as inactive, which makes it unavailable for future tests, but does not affect previously created tests and existing performance records.

Creating Quick Tests

You create a Quick Test if the test uses all or majority of questions in a question bank. A Quick Test takes the default properties, and contains a single test section that holds all questions (preselected) from the question bank. You can edit the test to modify the properties, for example to set the scoring option and mastery score, or to add or remove questions.

Note: You cannot use the quick test option to create a rule-based test, in which you display a number of questions extracted randomly from the question bank to the learner.

Test Creation

After you have a created question bank and added the required questions, you can create a test and define the test properties.

Selecting Specific or Random Questions at Run Time

For each test section, you can select exactly the questions you wish to add (pre-selected) or you can enter the number of questions and allow the test to extract a random selection of questions when a learner launches the test (rule-based). The default is pre-selected.

Randomizing the Sequence of Sections and Questions

You can choose whether multiple sections always occur in the same sequence, or in a randomized sequence each time learners launch the test. There is a similar option for sequencing questions within a section. However, you cannot randomize questions across more than one section.

Resuming Tests

Select the Is Resumable box for learners to resume a test if they previously exited or were logged out without submitting the test. Learners are taken to the page where they last left the test. If the time has expired for the test, learners are taken to the page that allows only submission of the test. The Resumable Instructions appear each time a learner resumes a test.

Assessment Types

Depending on your business requirements, you can create tests, surveys, and evaluations by selecting the appropriate option in the Assessment Type field. Typically, you create tests that are scored to measure candidates' knowledge and skills, and evaluations that are not scored to collect feedback on training courses.

Scoring a Test

The scoring options are percentage scored, or sum of item scores. If your test is rule-based (random questions at run time), be aware that the sum of item scores will vary if some questions have different points from others.

If you select either the percentage or sum scoring options, you can also set the mastery score to indicate the minimum pass mark for the test. For example, using the percentage scored option, enter a mastery score of 75 to indicate a 75 per cent pass mark. Using the sum of item scores option, enter a mastery score of 12, to indicate the learner must obtain at least 12 points to pass the test.

When you set a mastery score for the test, a learner passes or fails a test based on the results. If you do not define a mastery score, the application calculates the points scored, but marks the learner's test attempt as completed, rather than passed or failed.

Providing Feedback

By default, feedback is provided at the end of the test. You can change the feedback type to After Each Page to provide more frequent feedback. You can define the following feedback options:

• Select Questions Inherit Feedback if you have test questions that do not have defined feedback, and you want them to inherit (and display) the section or test feedback.

• Deselect Show Answer in Feedback to prevent learners seeing the correct answers when they review the feedback. This option is useful to encourage learners to retake the test. (The default is selected.)

• Enter text for Correct Response Feedback and Incorrect Response Feedback, to provide different feedback for the learners' responses.

• Enter Post Test Feedback to display a message to learners at the end of the test.

Limiting Test Attempts

You can limit the maximum number of times that a learner can attempt the test before being prevented from launching the test. You can also specify the minimum time that must elapse between each attempt.

Creating a Timed Test

To restrict the time allowed for a learner to complete a test, enter values for the duration properties of the test. If you specify a value for Time Allowed in Minutes, you can optionally set the additional duration fields. For example, if the time allowed is 30 minutes, you can enable the Time Remaining Alert, and display an alert five minutes prior to the end, so the alert appears after 25 minutes into the test.

Note: The timer resides on the server, not on the learner's client, so this message is only updated each time the browser page is refreshed. As long as the learner remains on the same page, the alert is not updated.

Section Management

A test must contain at least one test section and at least one question. You create test sections to organize the questions drawn from a question bank. If your test has multiple sections, you can specify the order in which the sections are displayed to the learner in the test. However, you can determine a section order only if you have selected the Fixed option in the Section Order field in the test properties. If the Random option has been selected for the Section Order field in the test properties, the test displays a different order of sections on each learner attempt

Deleting Sections

Once you make a test available to your learners, and at least one learner has taken the test, you cannot delete the test section (or the test), since this affects learner performance and test statistics on your production system. If one section becomes invalid, but the test itself is still available, make the section inactive, instead of deleting it. Unpublish a test to make the complete test unavailable to learners. Past performance and test statistics information remain in the system when you unpublish a test.

Test Previews and Delivery

The application provides a Preview option on several different screens, to preview questions at different stages, either when creating questions in a question bank, or when creating a test:

• Question Banks: Click Preview in the tabular list of question banks to preview all questions in a specific bank.

• Manage Questions: Click Preview in the tabular list of questions, within a bank, to preview an individual question.

• Test Section: Click Preview in the tabular list of test sections to preview the questions in a specific section of the test.

• Test Properties: Click Preview to preview all questions in the test.

Delivering Tests

You deliver a test to learners in two ways:

• Topic within a course: Create your test within the structure of an online course, in the content tree. A learner can access the test either directly from the overview in the course details page, or from the outline in the player after launching the course. A course can contain multiple tests (if the tests are grouped within one learning object).

• Separate course: Create your test as a top-level object within a folder in the content tree. The test becomes an individual playable object, for which you then create an offering in the catalog. When learners launch the course, the test appears in the player. A separate course usually consists of only one test.

Offering a Timed Test

You can alter the time of a test as a learner is currently playing the test. The Change Time option enables you to reduce the time allowed for a test, or allocate additional time, for example when a learner encounters a hardware problem during the test. This option is available only if:

• The test is timed and defined as resumable. You can create a timed test by setting the Duration properties during test creation. See: Creating a Timed Test

• The test is an offering in its own right, not a test topic contained within a course.

• The learner is enrolled in a class under that offering. The Change Time option applies only to an individual learner.

• The learner is currently playing the test. You cannot change the time of a test that the learner has not started or that a learner has already submitted.

Reviewing Test and Question Statistics

Statistics are available for you to analyze learners' performance for a given test or question. Statistics exist only after at least one learner has attempted the test or question.

Reviewing Test Statistics

For each test, the Test Statistics page provides a summary of learner scores for the selected test. Use the statistics to analyze both overall performance and individual attempt results. The page initially displays summary results in a tabular format, and provides an option to display a graph for each summary.

The summary statistic sections are:

1. Score Summary: Summarizes performance records for all learners. Each learner has one performance record that in turn is a summary of the learner's attempts. A performance record shows the score for the first Pass attempt, or the score for the first Fail attempt. The application updates a performance record from Fail to Pass (including the Pass score), if a learner subsequently passes the test. However, a performance record is never updated from Pass to Fail if a learner subsequently fails the test. A performance record does not change even if the learner score improves. A learner's performance record may show a Pass with a score of 60, even though the learner achieves a Pass with a score of 70 in a later attempt.

   The User Score Summary graph displays the number of users who achieved a specific performance score.

2. Learner Summary: Summarizes the total number of learners who launched the test, and the total number of performance records (one record per learner). The summary groups performance records as one of the following:

   • Passed: The learner completed the test and achieved the mastery score or better.

   • Failed: The learner completed the test but failed to achieve the mastery score.

   • Completed: The learner completed the test, but the test has no mastery score, or the scoring option is specified as not scored.

   • Incomplete: The learner started the test but exited the player without submitting the test.

   The table enables you to drill down to more detailed statistics for each learner. Click the Total Learners number to view all learners, or click the Passed number to view only those learners who passed the test. The drill-down table shows one performance record for each learner. Click an individual learner name to view all attempts for the learner. An individual attempt can show a higher score than the learner's performance record, since a performance record stores only the score for the first pass.

   The User Attempt Score Summary graph displays the score for each individual attempt number. Attempts are color coded to show the day of the attempt.

   In the table, click a specific attempt number to view the learner's individual attempt details for each question in the test.

   The Learner Performance Status graph shows the percentage of Passed, Failed, Completed or Incomplete performance records.

3. Attempt Summary: Summarizes all attempt records for all learners, not just performance records.

   The Monthly Attempt Summary graph shows the number of attempts made in each month.

4. Question Summary: Summarizes the total number of questions displayed to all learners in all attempts.

Reviewing Question Statistics

For each question bank, the Question Statistics page provides a summary of learners' performance for each question, across multiple tests. Statistics appear in both tabular and graphical formats. The graph shows Correct, Incorrect and Skipped totals as percentages of the total number of attempts for the question.