Concepts vs. Processes: Still More Thoughts on Khan Academy

Until Khan Academy attempts to differentiate between concept- and process-based learning, Sal Khan's instructional videos will continue to stand at odds with inquiry-based education.

Khan Academy is in the news again! Or maybe it never left... Ok, ok, I'm sorry for contributing yet another KA post to the education blogosphere (This is my third already, and I'm far from the worst offender), but this stuff's been on my mind a lot lately!

Recently, two math teachers posted a critique of a Khan Academy video, thus stoking the flames of an endless debate over the educational value of instructional videos. This video critique, dubbed Mystery Teacher Theater 2000, or #MTT2K, has received a lot of attention, and even spawned a contest to create the best KA critique. I'm proud to say that I've made my own #MTT2K video, which is embedded below.* Though Sal Khan's response to this criticism has been encouraging, I'm concerned that much of the debate surrounding Khan Academy obscures a subtler examination of the role that instructional videos should and should not play in a "revolution in education."

A lot of the Khan-bashing that gets tossed around is focused on aspects of Khan's videos that are unclear, poorly presented, or downright incorrect. Unfortunately, plenty of the KA videos can be criticized in this regard, but it's far from the majority, and Sal Khan's positive response to the #MTT2K project made it clear that he recognizes the benefit of rooting out and correcting such mistakes. As for the the gaffs, some fans of KA have said that Khan's occasional typos and stumblings make him a less intimidating tutor, and Khan is generally showered with praise for the clarity of his explanations. The majority of comments posted below his videos reveal as much. But for my money, the most severe criticism of Khan Academy has nothing to do with the clarity, or even the accuracy of a given video. Within an inquiry approach, clear and accurate explanations are actually a threat to the learning process.

Now, I freely admit that plenty of valuable information-gathering takes place through methods that aren't based in inquiry. For communicating the ins and outs of some accepted process, the instructional video medium is a fantastic way to create and store decent explanations. When I want to know how to apply some obscure filter in a photo-processing application, I don't spend much time performing experiments to arrive at the technique by inquiry. I go find an instructional video on YouTube that was made by some 13-year-old!! But truly process-based tasks are a tiny fraction of the learning that we're asking of our students. The great fear about Khan Academy is that it encourages students to see everything they're learning - addition, multiplication, algebra, calculus, free-body diagrams, conservation of energy, or even analyzing the actions and impulses of human beings caught up in a momentous event - as process-based tasks.

Is it unreasonably picky to insist on the sanctity of the inquiry process? 30+ years of Physics Education Research suggest that it isn't... The human mind is notoriously excellent at fitting in new explanations between the cracks of the things we think we know already, just so we don't have to throw out the old stuff. In my own contribution to the #MTT2K project, I tried to portray this phenomenon at work.

Admittedly, Khan took on quite a challenge in attempting to lecture about acceleration, a topic rife with nuance and levels of partially-correct understanding. The voice-over by the "student" shows how the video reenforces many common preconceptions, including but not limited to:

   • equating a clock reading (denoted by t) with a time interval (denoted by ∆t)
   • equating the direction of velocity with the direction of acceleration
   • misinterpreting common units of acceleration (m/s², or in this case, miles/s²)

Furthermore, Khan spends most of his lesson discussing unit conversion, a process-based task as fantastically mindless (and perversely satisfying) as painting a wall. Like wall-painting, it has to be done correctly, and a target instructional video could accomplish this instruction effectively if it wasn't folded into a lesson on acceleration. Indeed, Khan has made at least two videos (1, 2) that explicitly cover the subject of unit conversions, and together they've been watched over 200,000 times. Unfortunately, both of these videos ramble through the peripherally related topic of metric prefixes, fail to sufficiently demonstrate why multiplying by a "conversion factor" doesn't change the quantity represented, and do not contain examples of more complex conversions (How many m³/s are in a cm³/hr?), but these are subtleties compared to my main criticism of Khan Academy. We might be able to effectively offload to a video the task of teaching students to convert units correctly. (I couldn't find a video I'd want to use on Khan Academy today, but I might find it on Khan Academy someday.) However, there will never be a curriculum of instructional videos that builds up conceptual understanding of acceleration.**

There are more processes than just unit conversion involved in constructing a working model of acceleration, and instructional videos may have a role to play in students gaining familiarity with them. Using computer-graphing software is certainly one example. However, try to extend this list much further, and you see that making an explicit distinction between concept- and process-based tasks is pretty tricky. Is calculating the slope of a velocity-time graph process based? How about interpreting the meaning of this slope? How about linearizing a position-time graph? In any case, how can we tell if our video-curriculum has been effective? Purely process-based approaches to solving physics problems can be quite successful according to some measures, and assessments that truly discern correct conceptual understanding are a challenge to both develop and implement.

Luckily, our goal isn't to compartmentalize pieces of our curricula into "concepts" and "processes." The bottom line is that true learning requires students to actively make this distinction for themselves, and to approach solving new problems like a thoughtful human being, not a knowledgeable robot (damn those 100% success rate robots...). If this distinction is to be made by students, it has to made by teachers first, whether they're in person or online. So far, Khan Academy hasn't shown an interest in exploring this.*** Until they do, Khan's videos will continue to stand at odds with inquiry-based education.

*Though I made my video before I knew that there was going to be big prize money involved, it's fantastic that other teachers now have some more incentive to voice their opinion. Bring on the competition! Show us what you've got!!

**Do I truly believe that no videos will ever contribute to learning something conceptually? A definitive claim like this would require a rigid distinction between concepts and processes, which is impossible and sort of pointless. Regardless, I'd suggest that any conceptual understanding that comes from watching a lecture is a result of concept "construction" by the viewer, not "instruction" by the lecturer. Just as we've seen with research into the efficacy of in-person lecture courses, we can't rely on this concept construction taking place in most students.

***As I mentioned in my last post about KA, I got a chance to ask Sal Khan a question about the role of instructional videos in an inquiry process. He was somewhat dismissive of the criticism, suggesting that evidence against the benefit of instructional videos wasn't evidence against the benefit of HIS instructional videos. Specifically, he used an analogy about sugar pills and cancer research to suggest that his pills might just be the cure for cancer.

Advice for a New Teacher: Watch Others Teach

A couple days ago, a certain math teacher/blogger put out a "Call for Advice for New Teachers." The response has been amazing so far, and I'd wondered at first what I could possibly contribute... But I decided to mention something that's been on my mind a lot for the past couple years.

I'll keep it quick, because lots of folks have already given remarkable advice, and and my suggestion is pretty simple: Every now and then, or even just once or twice during the year, sit in on another teacher's class and watch them teach.

A couple posts have alluded to this idea with advice like "engineer friendships," and "observe, observe, observe" in the case of advice to a student teacher, but I think the call to observe is equally relevant for anyone in the classroom. This seems to be one of those things that everyone agrees would be helpful but somehow doesn't actually happen all that often. Some schools have organized peer observation programs, but most don't, so it's usually up to the individual to give up a prep period to do something that will never, ever seem pressing. But it's worth it...

It doesn't have to be a teacher in your subject area, or someone who works with students who are the same age as yours. Simply getting out of your own class and into a fresh environment can be amazingly eye-opening. Lots of unexpected things can happen when you venture beyond the walls of your own classroom. If you witness students you teach working in another environment, for example, this can help you see their strengths in a different light and appreciate all the other classes they're taking simultaneously to yours. If you see students in a class that comes before or after yours in the school's year-to-year sequence, you'll get some perspective on where they're headed or where they've been. Heck, it doesn't even have to be a teacher who's all that fantastic. Some of the most valuable observations I've made have involved watching someone and cringing to myself, "Why, oh why, are they doing it this way??" and then realizing immediately afterward that I do exactly the same thing. The goal isn't necessarily to see models of how to teach effectively, although this will certainly happen. In my experience, it simply about shaking things up and giving yourself some fresh perspective.

If this kind of thing isn't common at your school, you'll have to muster some courage to make it happen. Make friends with folks at your new school, then brooch the subject casually at lunch ("Hey, does anyone ever do peer observations here?") Then, if your new friends seems receptive, try suggesting that you might stop by their class some day, at their convenience, with plenty of advance notice!! I've found that usually people are amazingly receptive, and even excited to have a visitor. You might even inspire them to sit in on a few others' classes, who'll sit in on a few others' classes, who'll sit in on a few others' classes...

One Short-Lived Physics First Program: A Cautionary Tale

One short-lived implementation of Physics First at a New York City public school should serve as a cautionary tale of the challenge faced in convincing a local community that ninth graders can succeed at physics. The format of a Modeling Instruction summer workshop can establish a productive relationship between teachers to help take on this challenge.

Some time ago, I sat down to talk with the principal of a high school in New York City that opened in 2010 with a commitment to teach physics to all ninth graders. The decision to teach Physics First was one of many qualities that made this school unique in its geographical area, including an emphasis on the arts, interdisciplinary coursework, and a consistent focus on three essential questions: Who am I? Who do I want to become? How do I get there? The Physics First component, however, was a sticking point for many, from the administrators who approved the school's application to the parents who enrolled their children at the school. Many voiced skepticism that ninth graders could do physics, but the school's principal, herself a ninth grade physics teacher, assured them that Physics First could be successful. The administration selected a curriculum that was backed by promising research involving ninth graders and teachers underwent a week-long training session during the summer to prepare to use the method.

The ninth grade physics courses, however, got off to a rocky start.  As early as the initial training period, teachers felt that the chosen curriculum program lacked sufficient hands-on work to engage students. The program emphasized group problem solving with a heavy quantitative emphasis accompanied by a small component of direct instruction* involving interactive whiteboard technology. Teachers were encouraged to follow a predetermined script dictated by the developers of the program, and the training itself was lecture-oriented. When students indeed proved unreceptive to the approach, individual teachers tried to reorient the course to their own priorities, diverging independently from their common training experience in an attempt to improve their own class. 

Meanwhile, skeptics of the program looked for evidence of failure that would bolster their argument to convert to a conventional curriculum order. No other schools in the immediate area were teaching Physics First, and parents lacked a concrete measure for the success of the program. Most students wouldn't be sitting for their first state-standardized NY Regents exam until eleventh grade and parents were terrified that their children would fail this exam and be stuck without having fulfilled basic graduation requirements. Midway through the second year of implementation, this lack of direct evidence for the success of the program won out. The DOE stepped in, making the decision to abandon school-wide Physics First and removing the principal from the school completely.

How might things have gone differently at this school? Could anything have been done to set doubting minds at ease? I think that this story provides an important case study in examining what a Physics First program needs in order to be successful. In this case, the pressure to abandon Physics First was rooted in parents' mistrust that this non-traditional program would not meet students' needs, driven primarily by a concern over fulfilling testing requirements. Ironically, results from other public Physics First schools indicate that students do quite well on a standardized biology test when they take the test for the first time as Juniors (at least in part due to the fact that these tests are generally written to be taken by Freshmen). Even if this is confirmed at this school, no one will know until next June, when the test is given to the school's first ninth graders. But in an environment of high stakes testing, parents and students can't simply be asked to muster the patience to "wait and see" if such a program has been effective.

Any school planning to institute a Physics First program can expect that this decision is not going to get the benefit of the doubt from parents, students, or even faculty and administrators. Perhaps a gracious transition is more likely in an independent school, where parents might feel bound by a tuition to maintain faith in the school and its decisions. Private school students are not usually subject to external testing requirements, and if a family doesn't support a curriculum decision made by a school, they're free to take their child and their money elsewhere. But in the public school system, inertia rules. "You basically have to teach an existing class," the principal of this school told me. "New York State has defined the Regents classes, and [physics] means a very specific vision involving eleventh or twelfth graders. It's hard to do [anything different]." A larger movement toward Physics First, perhaps on a district level, might help reassure parents that their individual child won't be left out in the cold, but failed Physics First initiatives such as the program in San Diego in 2001 demonstrate that this reassurance will only go so far.

A cohort of teachers implementing a new Physics First program needs not only formal training in how to teach Physics First effectively, but time and freedom to develop unified goals and methods for a specific population of students. In interpreting this particular story, I've come to the conclusion that in order for a public school implementation of Physics First to be successful it has to meet a much higher bar than a traditional science program. Traditional physics courses that conform to parents' and administrators' expectations are simply awarded the benefit of the doubt even when the value of this status quo is deeply doubtful. The paradigm of a Modeling Instruction summer workshop suggests a means by which to lay the groundwork for implementing a program that's both informed by PER and responsive to the needs and concerns of the school community. Since Modeling Instruction is so visibly different from conventional physics teaching, individual teachers learn early in their exposure to Modeling that, regardless of their personal experience and expertise, they'll need to attend a workshop training in order to apply the method in their own classrooms. When a group of teachers in a school or district is implementing a Modeling curriculum together for the first time (as was the case at my first workshop last summer), many teachers from the same school have time during the workshop to share ideas, reactions, and come to some agreement on their collective goals for the course.

Although it's been said many times, many ways, effective classes are created by effective teachers! Likewise, effective curriculum has to foster teachers' ability to remain flexible and creative with the application of that curriculum to a specific student population. Training workshops are as much about developing a camaraderie and common language between cooperating teachers as they are about exposing teachers to new methods. As one ninth grade physics teacher at this school wrote to me, "In order to be effective, teachers need flexibility to break the rules if something isn't working. Nowadays the trust in teachers has diminished, causing classrooms to resemble more a preparation for standardized test centers than anything else." Physics First provides an opportunity to break this pattern, but only if the classes can convince local communities to give this unconventional sequence a chance. Teachers are the only people who can make that work, and to do it they need time, training, and the freedom to implement curriculum they're invested in.


* Anything that I've seen called "direct instruction" has seemed like a desperate attempt to hang onto lectures within a sea of research showing that they're simply not effective. Just like the speaker says in the video linked to here, "If you look at the trends in education today, the majority of schools are looking for scientifically based instructional programs." So... lectures work because they have to? Hmm... At least it provides for some fine comedic material!!