Good test prepping IS good learning

I've been in edtech for a while now, and the more I've read about assessment and learning research, the more I've realized something counterintuitive.

The relationship between assessment and learning is messier than most people think. Some educators, like those at Khan Academy, say we don't need separate tests at all, like if kids master material through practice, testing becomes pointless. That makes sense.

But here's what I found: the things that make mastery learning work are the same things that make good test prep work.

There's a difference between good and bad test prep. Bad test prep is drilling answer patterns without understanding, memorizing formulas you don't get, cramming the night before. That's not learning, it's gaming the system.

Good test prep is different. Spaced practice over weeks. Retrieving information without notes. Explaining your reasoning out loud. Finding the gaps in what you know. These are proven learning methods that also happen to improve test scores.

When I watched what happens during effective test prep, I noticed something. Kids aren't just reviewing material they already know. They're discovering that their understanding is shakier than they thought.

Take a kid who's been doing well in chemistry class. She understands balancing equations when the teacher walks through examples. But put her in front of a practice test, and suddenly she's staring at "C₃H₈ + O₂ → CO₂ + H₂O" with no idea where to start. The absence of her notes, the ticking clock, the forced retrieval - these constraints create a moment of truth. She thought she understood, but she was just following along.

Or consider the history student who can discuss the causes of World War I in class discussion but freezes when asked to write an essay about it in 45 minutes. Without classmates' ideas to build on, without the teacher's guiding questions he discovers that his knowledge is more fragmented than he realized.

These moments of cognitive dissonance - the gap between "I thought I knew this" and "I actually can't do this" - are where real learning begins. The constraint forces the confrontation with reality.

But what exactly creates these powerful learning moments? It's not just any feedback, it's feedback that forces students to confront the gap between what they think they know and what they can actually do.

The specific conditions matter: the inability to reference external supports, the requirement to articulate reasoning, the time pressure that prevents overthinking. These constraints aren't obstacles to learning, they're the mechanisms that make it happen.

Research confirms what we see in practice. Black and Wiliam's 1998 study found that frequent feedback produces learning gains with effect sizes between 0.4 and 0.7 - bigger than most educational interventions. An effect size of 0.4 means the average kid would achieve what only the top 35% achieve without that feedback.

Good test prep teaches exactly what we want kids to learn: reading carefully, organizing thoughts systematically, managing cognitive load, checking work efficiently, adapting when the first approach fails. These aren't test-taking tricks. They're thinking skills that work everywhere.

More importantly, good test prep teaches kids to take ownership of their thinking. When you can't ask for help or check with a friend, you have to rely on your own understanding. But what specifically creates that sense of ownership?

It's the forced decision-making. When there's no external help, every choice, which formula to use, how to structure an argument, whether an answer makes sense, becomes yours. You can't offload the cognitive work to someone else. You have to commit to your reasoning, even when you're uncertain.

This creates a different relationship with knowledge. Instead of borrowing understanding from others or following along with a teacher's explanation, you're building your own mental model. The struggle to retrieve information from memory, the effort to organize scattered thoughts into coherence, that cognitive load is what makes the knowledge yours.

When a student finally balances that chemical equation after struggling through several attempts, they don't just know the answer, they know they figured it out. That builds confidence and intellectual independence in ways that supported practice never can.

The challenge: preserving learning in an AI world

As I've been thinking about all this, I keep coming back to something that's been nagging at me about AI and education.

If the learning power comes from working alone, from forced decision-making, from owning your mistakes, from not being able to offload cognitive work, then what happens when AI is always available? If assessment drives learning, and AI disrupts how we assess, then we're disrupting the primary engine of learning itself.

I'm not sure how to preserve what makes test prep effective (the struggle, the ownership, the forced confrontation with gaps) when AI can handle much of the cognitive work.

If a student uses AI to help structure their essay, do they still develop the skill of organizing their own thoughts? If they use AI to check their math, do they still learn to catch their own errors? I honestly don't know.

The real question isn't "what can AI do?" but "how do we design assessments that create genuine intellectual work when AI is available as a tool?" But I'm struggling with whether that's even possible, or if we're fundamentally changing what learning looks like.

The constraint has always driven the creativity. Remove the constraint of "work alone from memory," and we need new constraints that force genuine intellectual work. The mechanism matters more than the medium.

What this might look like in practice

I've been thinking through what this could look like in practice. Imagine giving students an AI-generated analysis of photosynthesis with three deliberate errors. One factual mistake about chlorophyll, one logical gap in the process explanation, and one terminology mix-up.

The cognitive work shifts in interesting ways. Instead of blank-page paralysis, students have something concrete to evaluate. They still need to activate their understanding of photosynthesis, but the mental load feels different, more like debugging than creating from scratch.

Here's what this could mean for actual assessment design:

In chemistry, instead of banning AI, we give students an AI-generated solution to a stoichiometry problem with a subtle error in the mole conversion. Ask them to find the mistake and explain why it leads to the wrong answer. They still have to understand the underlying concepts to spot what's wrong, but the cognitive work shifts from calculation to evaluation.

In history, provide students with a ChatGPT essay about the Industrial Revolution and ask them to critique its argument structure and identify missing evidence. They're doing the same analytical thinking as writing their own essay, but the focus moves from content generation to critical analysis.

In mathematics, give students three different AI approaches to solving a calculus problem and ask them to explain which method is most efficient and why. The cognitive work becomes understanding mathematical reasoning rather than just executing procedures.

The key is designing tasks where the AI does some work, but the human still has to do the thinking that matters for learning. Students can't offload the decision-making that builds understanding, they just make different kinds of decisions.

So where does this leave us?

I started by watching kids struggle with exam prep and thinking there had to be a better way. The research convinced me that good test prep actually IS good learning, it creates the feedback loops and gap identification that drive real understanding.

But now I'm grappling with a bigger question: if assessment is the engine of learning, and AI is changing what we should assess, then we're not just facing a test prep problem. We're facing a learning design problem.

Think about that chemistry student who couldn't balance equations without support, or the history student who froze writing an essay alone. Those moments of realization "I thought I knew this but I actually can't do this" are where learning happens. The question is how to create those moments in a world where AI can handle much of the work.

Maybe instead of asking "how do we AI-proof our tests," we should ask "how do we design learning experiences that create genuine intellectual ownership?" Maybe the answer is assessments where students debug AI-generated code, or critique AI-written arguments, or solve problems where human judgment is essential.

The kids struggling with exam prep aren't going anywhere. But maybe what they're struggling with needs to evolve. We need better constraints that force genuine intellectual work, not just different tools.

I don't have all (well, any of the) the answers yet. But I think anyone designing learning experiences needs to ask: what creates those powerful moments where students confront the gap between what they think they know and what they can actually do? And how do we preserve that struggle, even as the tools around us change?