AI Course Creation Comes to Course Platforms: Check the Quizzes

AI course creation has moved inside the course platforms. In October, LearnWorlds released AI that turns an idea into a complete course, assessments included, and Thinkific let anyone in its new course builder have AI draft their quiz questions. Building a course just got faster. Whether it got better depends almost entirely on what you do with the draft, and the quiz questions are where we’d look first.

What happened

LearnWorlds rebuilt its pitch around AI making the course. On 14 October 2026, LearnWorlds announced that its LMS is now a “Living Learning System”, presenting 12 features across course creation, personalised learning, design, certification, commerce and operations. Its one-line summary: “AI can do more of the making, while you stay in the director’s seat” (LearnWorlds, 2026).

The features followed on 21 October. The headline one, complete course creation with AI, promises to “turn your ideas into a complete course, from the outline to the learning content”, covering eBooks, assessments, videos and other activities, with a step where you review detailed activity plans before the full content is generated. Alongside it came an AI assistant to “create or update exams, forms, and self-assessments”, available on all plans, AI narration, AI video through a connected Synthesia or HeyGen account, and a Learner AI Agent that learners can ask for explanations while studying. The learner agent isn’t available on the Starter or Pro Trainer plans.

Thinkific put AI quiz drafting in its course builder. Thinkific’s round-up of its September updates, dated 13 October 2026, says the new course builder is now where every new customer starts, “with quizzes and discussions built in”, and that “anyone in the new course builder can let AI draft their quiz questions”. The workflow it describes is short: “describe what you want to assess and AI drafts the questions. You edit and publish” (Thinkific, 2026). The update doesn’t say which plans include it, but Thinkific’s help centre lists its AI quiz generator as available on all paid plans (Thinkific Help Center).

We also checked Teachable‘s product updates and Kajabi‘s release notes for the same period. Teachable’s latest product post we could find is from earlier in the year, and Kajabi’s September release notes are listed without detail, so we found no comparable dated announcement. That doesn’t mean they have no AI features, only that October’s clearly dated news came from LearnWorlds and Thinkific.

Why it matters for people who build courses

Most of the commentary on AI course creation, including the pages that rank for the term, is about speed: a course in minutes rather than weeks. Some of it mentions that AI output needs checking. Almost none of it asks whether the questions an AI writes are good practice for learners, which is the part that matters most.

Quizzes aren’t decoration. Retrieval practice (recalling information from memory rather than re-reading it) is one of the better-supported ways to make learning stick, and in most self-paced courses the quiz is where it happens. If the AI writes the quizzes and nobody rewrites them, the AI has quietly designed the most important learning activity in the course.

What the evidence says

The best evidence on AI-written questions comes from medical education, where exam questions are written carefully and tested with real candidates. It points in a consistent direction.

In a 2025 study in Hong Kong, Law and colleagues gave 24 doctors preparing for an emergency medicine exam two sets of 100 multiple-choice questions: one written by ChatGPT-4o, one by humans (Law et al., 2025). The AI questions were easier (mean difficulty index 0.78 against 0.69, where higher means more candidates answered correctly) but separated stronger and weaker candidates about as well. Expert reviewers found more factual inaccuracies in the AI set (6% against 4%) and more irrelevant questions (6% against none), and the AI questions mostly tested lower-order skills such as recall, while the human ones did better at higher-order skills. The AI saved a lot of time: 24.5 person-hours against 96. The authors’ verdict:

A 2026 systematic review and network meta-analysis by Riehm and colleagues, covering studies of AI-generated multiple-choice questions in health professions education published up to November 2024, found that ChatGPT-4 produced questions of similar quality to human ones on relevance, clarity and the quality of the wrong options (distractors), while an older open model, Llama 2, did worse (Riehm et al., 2026). The authors rated the certainty of that evidence as very low.

Why does the lean towards recall matter? Because the kind of question a learner practises with shapes what they can do afterwards. In Pooja Agarwal’s experiments with college and middle school students, retrieval practice consistently improved delayed test scores, but quizzes made only of fact questions did not improve performance on higher-order questions, while higher-order and mixed quizzes did (Agarwal, 2019). A course whose quizzes all ask “what is X?” will produce learners who can say what X is, and not much more.

Two limits apply. The question-writing studies are mostly in medicine, with expert-written comparison questions, and none measured what learners went on to learn from AI questions. And the platforms’ tools may behave differently from the general models tested. We’d treat the pattern (good-looking drafts, too easy, too factual, occasionally wrong) as early evidence, and a sensible default to check against.

What we’d do

1. Use the AI for the first draft, not the last. Both platforms describe a review step. Make it a real one. Budget the time the AI saves for reading every question.
2. Check each question against an objective. If your objectives say learners will “apply”, “compare” or “diagnose”, a recall question doesn’t test them. Our guide to writing learning objectives shows how to match the two.
3. Rewrite recall into application. Turn “Which of these is the definition of X?” into a short scenario: “A learner does this. What should happen next, and why?” Keep a few fact questions where facts genuinely matter, and mix in the rest.
4. Fact-check like an editor. In the medical study, 6 in every 100 AI questions had a factual inaccuracy. In a 20-question quiz, that’s about one to find. Check the answer key too.
5. Make questions harder than feels comfortable. AI drafts lean easy. A quiz that everyone aces on the first try gives learners little practice and you little information.
6. Keep the design decisions human. What to teach, in what order, and how learners show they can do it are the decisions in our online course design checklist. An AI can fill in a plan; it can’t tell you whether it’s the right plan.

Quiz drafting with AI is a great use of the technology, because AI is particularly good at reviewing copious amounts of transcripts, course information and documents. It’s a great time saver when finding ways to reinforce learning. It’s just important that the instructional designer doesn’t lean on AI for building foundational learning units, as AI has a tendency to use tired examples, particular turns of phrase, and diminishes the effectiveness of human led training, where the intent of the instructor is felt by the student.

What to watch next

On 31 October we wrote about learners bringing their own AI tutors to courses, in AI in eLearning this term. This is the other side: the platforms putting AI on the builder’s side of the screen. The useful signal from here won’t be more generation features. It will be platforms that let you set the level of the questions their AI writes, show question-level data so you can see which drafts are too easy, and studies that measure what learners can do after practising with AI-written questions rather than how the questions look.