An Online Course Design Checklist, Grounded in the Research

An online course design checklist should test whether a course will help people learn, not just whether it has the usual parts. The checklist below covers seven areas: the brief, objectives, assessment, practice, feedback, content and access, and the pilot. Each item says what to check, gives an example, and, where the research is clear, points to the evidence behind it.

Search for this topic and you’ll mostly find university checklists. They’re careful and useful, but they’re written for faculty working inside a learning management system, and most of their items ask whether something exists: a welcome message, a syllabus, contact details, a rubric. Few say why an item matters or what the evidence is. If you’re building a self-paced course on your own, you need a shorter list that focuses on the decisions that decide whether learners can do the thing you’re teaching.

How to use this checklist

Run it three times, not once:

1. Before you build, on the plan: sections 1 to 3.
2. After your first module, on real material: sections 4 to 6.
3. After a pilot, on real learners: section 7.

A checklist used only at the end of a build catches typos and broken links. Used at the start, it catches the expensive problems: a course aimed at the wrong gap, or a quiz that tests something the lessons never practised.

To keep the items concrete, we’ll follow one example: a photographer building a short self-paced course that teaches beginners to shoot in manual mode. It’s an invented example, but it’s typical of the courses independent creators build.

1. The brief

• You can name the learners in a sentence: who they are, what they already know, how much time they have.
• You can name the gap. What can’t they do now, and why? Lack of knowledge, lack of practice and lack of a routine need different courses.
• A course is the right fix. If the problem is a missing checklist or a confusing tool, a one-page guide might do the job better.

For our photographer, the brief might reveal that most learners already know what aperture and shutter speed are. What they lack is fluency: they freeze when the light changes and switch back to automatic. That points to a course built around short shooting exercises, not long explanations.

This is the analysis phase of the ADDIE model, and it’s the step creators skip most often.

2. Objectives you could watch someone meet

• Every objective names something observable. “Understand exposure” fails. “Set a correct exposure in manual mode for a backlit portrait” passes.
• Each objective is small enough to practise in one lesson or module.
• The objectives add up to the goal in the brief, with nothing important missing and nothing there just because the expert finds it interesting.

3. Assessment that matches the objectives

• Every objective has an assessment, and every assessment tests an objective.
• The final assessment looks like the real task. For the photographer, that’s submitting three photos taken in manual mode in different light, not a multiple-choice quiz on the exposure triangle.
• Learners see the criteria in advance: what a good submission looks like, ideally with an example.
• You designed the assessment before writing the lessons. Working backwards from the task keeps lessons focused on what learners need to do.

The top-ranking university checklists, including those from Carnegie Mellon’s Eberly Center and UC San Diego, both put objectives and the assessments that test them near the top of the list, for good reason: everything else in the course hangs off them.

4. Practice spread through the course

This is where the research is strongest, and where most self-paced courses are weakest.

In 2013, Dunlosky and colleagues reviewed the evidence for ten common study techniques, judging each on how well it generalised across learners, materials and tasks. Two were rated high utility: practice testing (recalling information or skills from memory, as in a quiz) and distributed practice (spreading study out over time rather than cramming it into one session) (Dunlosky et al., 2013). Several familiar techniques fared badly.

The review was about techniques students choose for themselves, but it translates directly into course design. A course that is mostly videos to watch is, for the learner, mostly rereading.

• Every module ends with practice, not just content: a short quiz, an exercise, a task.
• Practice asks learners to produce, not just recognise. Where you can, ask them to recall, explain, solve or perform rather than pick from four options.
• Earlier material comes back later. Include a few questions or exercises from previous modules in each new one, so practice is spaced out across the course.
• Quizzes are low-stakes and repeatable, so learners use them to practise rather than to avoid failing.

For the photographer, each module might end with a shooting exercise in a different kind of light, and every new exercise includes one situation from an earlier module.

5. Feedback that tells learners what to do next

• Every practice task gives feedback, even if it’s automated.
• Feedback explains, not just scores. “Incorrect” teaches little. “This photo is underexposed: the subject is backlit, so open the aperture or slow the shutter” teaches the task.
• Learners get a chance to use the feedback: a retry, a second attempt, a similar question.

A 2020 meta-analysis of educational feedback research, covering 435 studies, found a medium overall effect of feedback on learning (d = 0.48), and found that the effect depended heavily on the information the feedback carried. The authors report that simple forms of reinforcement and punishment had low effects, while feedback containing information about the task and how to do it was the most effective (Wisniewski et al., 2020). The studies varied widely in setting and learners, so the size of the effect in any one course will vary too; the direction is the useful part.

In practice, write your quiz feedback when you write the questions. For each wrong answer, say why it’s wrong and point back to the lesson that covers it.

6. Content that’s short, organised and accessible

• Videos are short. Split long explanations at natural breaks.
• Each lesson has one job, named in its title, and learners can always see where they are and what comes next.
• Format follows content, not learner preference: demonstrations for physical skills, diagrams for spatial ideas, worked examples for procedures.
• Every video has captions, and every image that carries meaning has alt text.
• The course works on the devices learners will use, including a phone.

On video length, the best-known evidence comes from Guo, Kim and Rubin’s study of 6.9 million video viewing sessions across four edX courses. Median engagement time was at most about six minutes, however long the video (Guo et al., 2014).

Note what that study measured: how long people watched, not how much they learned. A six-minute video that’s never practised still won’t stick. But a video people stop watching halfway through can’t teach its second half.

On format, resist the idea of designing for “visual” or “auditory” learners. A review by Pashler and colleagues concluded that “there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice” (Pashler et al., 2008). Choose the format the content needs.

On access, captions are a baseline, not an extra. The W3C’s Web Content Accessibility Guidelines make captions for prerecorded video a Level A requirement, the most basic level of conformance (W3C, WCAG 2.2). Captions help deaf and hard-of-hearing learners first, and they help anyone watching on a train or in a shared office.

7. A pilot, then evaluation

• One complete module was tested with real learners before the rest was built.
• A pilot group took the whole course before public launch: five to ten people, in return for honest feedback.
• You measure whether learners can do the task, using results on the final assessment, not only a satisfaction survey.
• You know where people stop. Check which lessons lose learners and which quiz questions most people get wrong.
• Every finding becomes a fix in the right place: a confusing question is a content fix; a learner who didn’t need the course is a brief problem.

For the photographer, the pilot might show that most learners pass the final task but nearly all of them stall on the same indoor-lighting exercise. That’s a specific, cheap fix, found before hundreds of people hit it.

The checklist at a glance

Area	The check	When
Brief	Learners, gap and goal written in one page	Before building
Objectives	Each one observable and small enough to practise	Before building
Assessment	Matches the objectives and looks like the real task	Before building
Practice	In every module, recall not recognition, earlier material revisited	First module
Feedback	Explains why and what to do next, with a chance to retry	First module
Content and access	Short videos, one job per lesson, captions and alt text	First module
Pilot and evaluation	Tested with real learners; measures the task, not just satisfaction	After pilot

What the platform checklists miss

The university checklists that rank for this topic are thorough on presentation and policy: welcome modules, contact details, grading schemes, institutional support links. Those things matter in a for-credit course. For an independent course, three gaps stand out:

1. They rarely mention spacing. Practice is usually checked as “present”, not “spread out”.
2. They treat feedback as logistics, such as how fast it arrives, rather than what it contains.
3. They start after the plan is made. None of the four we read asks whether a course is the right fix in the first place.

That’s not a criticism of their authors, who were writing for a different job. It’s why a creator needs a different list.

What to do next

Before you record anything for your next course, write the one-page brief and the final assessment. Then check them against sections 1 to 3. If the assessment doesn’t look like the real task, fix it now: it’s the cheapest change you’ll ever make to that course.