[0:02] This video will share some best practices for crafting effective multiple-choice questions for online assessments. While this type of activity is not always ideal for the online environment, there are certainly times when it can be necessary, or even simply helpful. Let’s consider some strategies instructors may use to get the most out of online multiple-choice quizzes.

[0:27] The first step concerns quiz settings. As with other online activities, it is a good idea to provide thorough, detailed instructions about how the multiple-choice assessment will work. You might even find it worthwhile to create a video with instructions for students about where to locate the quiz, what they can expect to see on it, where they can try practice problems, etc.

[0:53] It is also a good idea to use a test bank. Learning management systems like Canvas allow instructors to upload as many multiple-choice questions as they would like, and students can be asked to answer some subset of the larger pool. For instance, if an instructor wishes to make a ten-question quiz, they could upload twenty or twenty-five questions, and students will be randomly assigned some combination of ten of them. This ensures that students all take slightly different quizzes, but since they are all randomly drawn from the same pool, the quizzes will work out to be equally rigorous.

[1:34] It is also possible for learning management system software to randomly arranged the answer options in multiple-choice questions. The right answer does not change, but for one student the correct answer may be “A,” while another student viewing the same question may see the correct answer in the “C” position.

[1:54] There is no hard and fast rule on this point, but students appreciate when instructors set quizzes in Canvas to only present one question at a time. Instructors can decide whether to permit students to go back and reconsider previous answers. Unless there is a compelling reason not to do so, such as a question at the end of a quiz containing information that could answer previous question, it is generally best to allow students to move back and forth through all questions if they so desire. This allows students to move on from a question that is particularly difficult and return to it after spending time on other items.

[2:35] Finally, it is a good idea to routinely update quiz banks. It may not always be feasible to replace all question every term, but is it a good habit to replace at least a sizable minority of question from term to term.

[2:51] What about the questions themselves. Here, too, a few best practices can boost the efficacy of the assessment. One helpful approach is to focus on higher-order questions that ask students to engage in rich cognitive tasks like applying or analyzing information. For example, instructors may place students in a hypothetical situation and ask them to describe what they would advise a client to do in a situation, or what route through the mountains they would take, or what colors they would use to create a piece of art.

[3:26] At the same time, it is best to avoid lower-order questions that ask students to recall information, or identify easily “Googlable” items like dates and names. In fact, one helpful strategy is to place each question into Google to see the common search results. If you are feeling cheeky, you might include as wrong answer options some of the common incorrect answers from the Google search.

[3:57] Consider this example of a fairly common statistical question in social science research methods courses. Let’s subject it to the Google test.

[4:10] When we copy and paste the question into the search engine, we see numerous website appear that immediately present the answer to the question. In fact, a few top results actually show the answer in the result preview itself! This does not make for a good quiz question if the goal is to assess whether students in an online course have committed to memory information about standard confidence levels.

[4:35]So how might we assess knowledge of polling instead? Here is one way to do so. This quiz question present a piece of polling information to students and asks them to identify the more accurate newspaper headline describing the poll results. Students must recognize the polling information, apply it correctly, and identify what the poll is actually saying. In this case, the tempting answer is the first one; however, discriminating students using the margin of error properly will settle on the second answer option as the best choice. This multiple-choice question actually requires students to know and use quite a bit of information, yet scoring it is quite simple.

[5:27]Let’s consider a helpful guide for thinking about online multiple-choice assessment. This table lists numerous verbs related to different levels of Bloom’s taxonomy. Lower-order activities are at the top of the page and higher-order activities are at the bottom. When designing multiple-choice assessment activities for the online environment, it is best to focus on the middle- and bottom-part of the table. One great strategy is to pick a variety of verbs from this list and match them with concepts from the course. From there, design a creative quiz item using the concept-verb pairs you created!

[6:07] In closing, let’s consider one more example of a higher-order multiple-choice assessment item, but this time we will add an important reminder. Here is a quiz question that requires student to identify which political candidate would win an election based on a certain rule for counting votes. As before, this requires some fairly heavy cognitive lifting, with students proceeding through several steps to reach the right answer.

[6:37] At the bottom is the student learning outcome associated with this quiz question. This is an important point to emphasize: with all attention we rightly place on designing creative and effective instruments, it is important not to lose sight of the fundamental goal of any assessment: does it align with course objectives?