New Media Technology multi-media Evaluating Your Tutorial, Usability, Evaluating Instruction

Evaluating Your Tutorial

image by freeimages.co.uk

Evaluation is a crucial step in the instructional design process. Evaluation allows you to con- tinuously increase the effectiveness of your instruction and hone your design skills. Since your instruction is an online tutorial, you’ll face unique challenges in evaluating your work—but don’t be discouraged! You will likely learn as much from the evaluation process as you did throughout the planning and production stages.

Your tutorial has elements of both a Website and instructional material. To accommodate these dual purposes, we’ll look at two different evalua- tion processes, one that examines the usability of Web resources, and one that assesses the success of the instruction itself.

Usability

In recent years, usability has taken center stage in the world of Web development. Becoming familiar with standard usability heuristics (such as those published by Jakob Nielsen), can help you make good design decisions through- out the planning and production processes.

Remember, usability encompasses a wide range of Website characteristics such as ac- cessability for the disabled, visual and syntactic organization, navigation and labeling, and many other distinct features of interaction. When you test the usability of your tutorial, do it in multiple browsers and platforms--the most basic test of usability is whether or not your site can be seen by Mac and PC users with a variety of browsers.

The best way to test your finished tutorial for us-

ability is to observe first-time users navigating your page or site and accessing the instructional materials. Don’t worry about the success of the instruction itself while you test for Usability. First determine whether or not users understand how to use the tutorial and can do so successfully. Usability problems can create false negatives when you test the efficacy of your instruction; it’s best to eliminate lingering usability problems before evaluating the instruction itself.

Evaluating Instruction

Instruction is generally evaluated using learner assessments of some kind. Assessment may include an interview, a skills demonstration or a written test. By carefully planning and choos- ing participants for the evaluation, you can

determine whether or not users of your tutorial are learning the skills you are trying to teach.

Be selective when choosing evaluation partici- pants. Don’t use staff members or even regular patrons of your library to test library instruction unless they are the intended learners. While it is tempting to use the resources close at hand, you’ll be surprised at the fresh perspective and ideas that outsiders will bring to the project.

When designing the test itself, use the instruc- tional goals you developed in the planning stage. Imagine ways that your learner could demonstrate newfound competencies; whether you use a written skills test, a personal dem- onstration or a personal interview, be sure to maintain a consistent format for each evaluator. Finally, remember that you are testing your in- struction—not the evaluators!

p. 13