Concepts vs. Processes: Still More Thoughts on Khan Academy

Until Khan Academy attempts to differentiate between concept- and process-based learning, Sal Khan's instructional videos will continue to stand at odds with inquiry-based education.

Khan Academy is in the news again! Or maybe it never left... Ok, ok, I'm sorry for contributing yet another KA post to the education blogosphere (This is my third already, and I'm far from the worst offender), but this stuff's been on my mind a lot lately!

Recently, two math teachers posted a critique of a Khan Academy video, thus stoking the flames of an endless debate over the educational value of instructional videos. This video critique, dubbed Mystery Teacher Theater 2000, or #MTT2K, has received a lot of attention, and even spawned a contest to create the best KA critique. I'm proud to say that I've made my own #MTT2K video, which is embedded below.* Though Sal Khan's response to this criticism has been encouraging, I'm concerned that much of the debate surrounding Khan Academy obscures a subtler examination of the role that instructional videos should and should not play in a "revolution in education."

A lot of the Khan-bashing that gets tossed around is focused on aspects of Khan's videos that are unclear, poorly presented, or downright incorrect. Unfortunately, plenty of the KA videos can be criticized in this regard, but it's far from the majority, and Sal Khan's positive response to the #MTT2K project made it clear that he recognizes the benefit of rooting out and correcting such mistakes. As for the the gaffs, some fans of KA have said that Khan's occasional typos and stumblings make him a less intimidating tutor, and Khan is generally showered with praise for the clarity of his explanations. The majority of comments posted below his videos reveal as much. But for my money, the most severe criticism of Khan Academy has nothing to do with the clarity, or even the accuracy of a given video. Within an inquiry approach, clear and accurate explanations are actually a threat to the learning process.

Now, I freely admit that plenty of valuable information-gathering takes place through methods that aren't based in inquiry. For communicating the ins and outs of some accepted process, the instructional video medium is a fantastic way to create and store decent explanations. When I want to know how to apply some obscure filter in a photo-processing application, I don't spend much time performing experiments to arrive at the technique by inquiry. I go find an instructional video on YouTube that was made by some 13-year-old!! But truly process-based tasks are a tiny fraction of the learning that we're asking of our students. The great fear about Khan Academy is that it encourages students to see everything they're learning - addition, multiplication, algebra, calculus, free-body diagrams, conservation of energy, or even analyzing the actions and impulses of human beings caught up in a momentous event - as process-based tasks.

Is it unreasonably picky to insist on the sanctity of the inquiry process? 30+ years of Physics Education Research suggest that it isn't... The human mind is notoriously excellent at fitting in new explanations between the cracks of the things we think we know already, just so we don't have to throw out the old stuff. In my own contribution to the #MTT2K project, I tried to portray this phenomenon at work.

Admittedly, Khan took on quite a challenge in attempting to lecture about acceleration, a topic rife with nuance and levels of partially-correct understanding. The voice-over by the "student" shows how the video reenforces many common preconceptions, including but not limited to:

   • equating a clock reading (denoted by t) with a time interval (denoted by ∆t)
   • equating the direction of velocity with the direction of acceleration
   • misinterpreting common units of acceleration (m/s², or in this case, miles/s²)

Furthermore, Khan spends most of his lesson discussing unit conversion, a process-based task as fantastically mindless (and perversely satisfying) as painting a wall. Like wall-painting, it has to be done correctly, and a target instructional video could accomplish this instruction effectively if it wasn't folded into a lesson on acceleration. Indeed, Khan has made at least two videos (1, 2) that explicitly cover the subject of unit conversions, and together they've been watched over 200,000 times. Unfortunately, both of these videos ramble through the peripherally related topic of metric prefixes, fail to sufficiently demonstrate why multiplying by a "conversion factor" doesn't change the quantity represented, and do not contain examples of more complex conversions (How many m³/s are in a cm³/hr?), but these are subtleties compared to my main criticism of Khan Academy. We might be able to effectively offload to a video the task of teaching students to convert units correctly. (I couldn't find a video I'd want to use on Khan Academy today, but I might find it on Khan Academy someday.) However, there will never be a curriculum of instructional videos that builds up conceptual understanding of acceleration.**

There are more processes than just unit conversion involved in constructing a working model of acceleration, and instructional videos may have a role to play in students gaining familiarity with them. Using computer-graphing software is certainly one example. However, try to extend this list much further, and you see that making an explicit distinction between concept- and process-based tasks is pretty tricky. Is calculating the slope of a velocity-time graph process based? How about interpreting the meaning of this slope? How about linearizing a position-time graph? In any case, how can we tell if our video-curriculum has been effective? Purely process-based approaches to solving physics problems can be quite successful according to some measures, and assessments that truly discern correct conceptual understanding are a challenge to both develop and implement.

Luckily, our goal isn't to compartmentalize pieces of our curricula into "concepts" and "processes." The bottom line is that true learning requires students to actively make this distinction for themselves, and to approach solving new problems like a thoughtful human being, not a knowledgeable robot (damn those 100% success rate robots...). If this distinction is to be made by students, it has to made by teachers first, whether they're in person or online. So far, Khan Academy hasn't shown an interest in exploring this.*** Until they do, Khan's videos will continue to stand at odds with inquiry-based education.

*Though I made my video before I knew that there was going to be big prize money involved, it's fantastic that other teachers now have some more incentive to voice their opinion. Bring on the competition! Show us what you've got!!

**Do I truly believe that no videos will ever contribute to learning something conceptually? A definitive claim like this would require a rigid distinction between concepts and processes, which is impossible and sort of pointless. Regardless, I'd suggest that any conceptual understanding that comes from watching a lecture is a result of concept "construction" by the viewer, not "instruction" by the lecturer. Just as we've seen with research into the efficacy of in-person lecture courses, we can't rely on this concept construction taking place in most students.

***As I mentioned in my last post about KA, I got a chance to ask Sal Khan a question about the role of instructional videos in an inquiry process. He was somewhat dismissive of the criticism, suggesting that evidence against the benefit of instructional videos wasn't evidence against the benefit of HIS instructional videos. Specifically, he used an analogy about sugar pills and cancer research to suggest that his pills might just be the cure for cancer.

Continue reading →

Khan Academy II: Discussions and "Khanversations"

"Khan Academy" style instructional YouTube videos could be more effective for introductory physics if they used a discussion model rather than a lecture model.

I had a fine time last week at the WNET Channel 13 Celebration of Teaching and Learning (which consisted of about 30% substance, 20% patting teaching on the back for doing "such an amazing job," and 50% advertising), and I wanted to follow up on the post I wrote about Khan Academy.

Over the course of the day, I saw Sal Khan (the Silicon Valley superstar shown in the camera-phone screens to the left) give his standard talk, and then follow it up with an hour-long question and answer session. In general, I came away convinced that Khan's heart is in the right place, and that Khan Academy strives to be far more than a YouTube channel. The goal of Khan Academy, he said numerous times, is to off-load a number of tasks traditionally done by teachers in order to free up the teacher's time to do more valuable things. During the Q&A, I got a chance to ask Khan essentially the questions that I posed in the last post: What is the role of an explanatory video when we know that clear and concise explanations can be counterproductive to student learning? His answer was basically that students should have access to whatever resources that might be helpful to them, and they're taking seriously their responsibility to measure the effectiveness of the videos to identify which ones aren't working. Here's a quote from his response:

When I think about my own learning, there are some times when I learned something through the experiential, where finally when I had to write a program when I was doing some computer graphics, trigonometry finally kicked in... But for some things, you know, especially when I was doing higher level math, it really sometimes was a friend in a coffee shop giving me a clear and concise explanation. And I was just like, "Wow, that really hit the spot. That was really much better than what was in the book, and that got me through my stumbling block."

I agree with what Khan is saying here, but this response reveals a slightly simplistic view of how learning works. I can't deny that clear and concise explanations from friends or teachers have gotten me through some tricky spots as well. However, I'd also suggest that hearing those explanations in clear and concise terms sometimes didn't actually help me as much as other approaches might have. Precisely because I was hand-fed exactly what I needed to fill in the gaps in my understanding at that moment, a few days or weeks later, those gaps sometimes returned.

When I think about what Khan Academy videos might look like if they were truly out to correct student misconceptions about, say Newton's Third Law, I imagine something more like the "dispute between students" prompts you find in Lillian McDermott's Physics By Inquiry books (see my previous post on this topic). In the Khan Academy model, picture a "Khanversation" between two voices, in which both individuals make arguments supported by diagrams to support a claim their view is consistent with observations in the natural world. This approach would provide opportunities to bring common misconceptions out into the open and model effective argumentation for students as they practice these concepts and skills in their classroom.

In a 2010 review paper in Science, Stanford School of Ed Professor Jonathan Osborne calls attention to a great irony in many science classes - traditional science teaching fails to develop the skills of argumentation and debate that are at the heart of the way science actually operates. Not only do student-centered teaching methods help to develop these essential skills, they also facilitate learning of science concepts far more effectively. Osborne writes: "Learning is often the product of the difference between the intuitive or old models we hold and new ideas we encounter. Through a cognitive process of comparison and contrast, supported by dialogue, the individual then develops new understanding. Consequently, learning requires opportunities for students to advance claims, to justify the ideas they hold, and to be challenged." We should be teaching our students first and foremost how to navigate their way through this process, as this is a skill that will be far more relevant to them than any science concept. (excepting, of course, Newton's Third Law...)

One of the most productive aspects of whiteboarding is that students are expected to formulate a verbal argument to support their answer, and present this argument to the teacher and their peers. Not only does this give a teacher instant access to their students' reasoning, but the students themselves are constantly exposed to effective and ineffective arguments. What role might other methods play in this process? I have tried to use handouts to structure and spur dialogue between students, but I've never gone so far as to upload such a dialogue to YouTube. At first glance, however, this possibility seems intriguing.

Continue reading →

Active Physics and Inquiry

An independent school in New York City provides an excellent example of a successful application of the Active Physics curriculum, but aspects of Modeling Instruction could have potential to make the course even more dynamic.

Active Physics is a project-based curriculum with a conceptual focus, designed to be used with ninth graders. Active Physics groups concepts by themes, such as "Communication," "Sports," or "Home," in an attempt to make the physics more relevant to students' daily life. The work done in each unit culminates in a "Chapter Challenge," where students must apply their knowledge to solve a real-life problem. One independent school in New York City has been using the Active Physics curriculum since 1994 as the foundation of a physics course for all ninth graders. When I visited this school, students were studying the efficiency of various methods of heating water, and were just about to begin the "Chapter Challenge" of selecting appliances to meet the basic needs of an average family, capable of being powered by wind-generator with an output of only 2400W.

When class began, students were seated in lab groups, discussing a question from their textbook: "Are high-efficiency appliance worth the added cost?" Students' responses reflected a common misconception - conflating the efficiency of an appliance with an assessment of its overall quality: "Well, yeah they're worth it... they're better." "They're more durable, work faster, and just work better in general." When students were asked to present the results of their discussion, only one group in three appreciated the more subtle implications of the concept of efficiency, stating, "A higher efficiency appliance will make up for its cost with less power used over time," but even this group was confused by the difference between the terms "power" and "energy." The stage was set for an inquiry-based activity to root out the would root out these misconceptions and lead students to a more sophisticated understanding of the concepts of energy, power, and efficiency.

The lab activity for the day consisted of heating up a beaker of water on a hotplate. The procedure steps outlined in their textbook were summarized on a projector: "Measure: 150mL of water, initial and final temp of water, measure time appliance is on (increase temperature by 20˚C)." After a brief discussion of how to use the equipment, students got to work carrying out these steps. They made a few potentially problematic procedural choices along the way (measuring water volume with a beaker rather than a graduated cylinder, plugging in the hotplate before starting their stopwatch, resting the thermometer against the bottom of the beaker, for example), but the teacher caught most of these and gave suggestions for improvements when he felt it was necessary to do so. In class discussion, students struggled with how to use the values they'd measured to make the required calculation of efficiency, but the teacher coached them through the process (partly by referring them to a similar activity done a few days earlier with immersion heaters):

Teacher: "Who remembers how we calculate the thermal energy gained by the water?"

Student: "Was that the thing that was 4.18...?"

Teacher: "Yes, we need the specific heat of water. Anything else?"

Over the course of the discussion, each group eventually arrived at calculations that basically agreed with one another, confirming an efficiency of about 10%.

While watching students carry out this activity and discuss the correct method for calculating efficiency, I tried to imagine what the same basic procedure would look like using a whiteboarding approach. Students might start the lab by brainstorming steps they'd take to to collect whatever data they felt were relevant to a calculation of efficiency, then writing these steps on a whiteboard and presenting them to the class for discussion. Once students had carried out these steps with their lab group, they could attempt a calculation of efficiency (again, on a whiteboard), and discuss as a class whether the calculations they'd made were relevant to the central question of the efficiency of appliances. Different groups could even use different methods of heating the water: an immersion heater, a hotplate, a microwave...

I emailed a prominent advocate of Modeling Instruction to ask about crossover between Active Physics and Modeling Instruction, and she told me that "Active Physics and Modeling Instruction don't go well together." Modeling Instruction is about developing basic models for the most fundamental interactions in physics, whereas the projects in Active Physics tend to highlight more complex applications of these concepts: efficiency of electric appliances, acoustic properties of instruments (watch your ears...), how to build a DC motor or generator, etc. Both of these approaches have merit, and it seems to me that there's a lot to be gained in exposing students to aspects of both. That is, a whiteboarding approach might have avoided the more "cookie-cutter" aspects of this particular lab activity on heating water (and probably brought misconceptions to the forefront more effectively), and a project-based "Chapter Challenge" in a Modeling course might give students a better appreciation for how even the simplest models they develop can be applied to their daily life.

In my observations, I've noticed a trend among teachers of Physics First: in the absence of a single universally-accepted ninth grade physics curriculum, teachers tend to pick and choose aspects of various programs that appeal to them. This dynamism is healthy and exciting, but there is something particularly thrilling about the momentum that has been building around Modeling Instruction. A lot of aspects of Modeling just feel like the right way to teach physics: whiteboarding, student-designed experiments, modeling phenomena with multiple representations, and it's tremendous to see the Physics First movement marching forward hand-in-hand with Modeling Instruction. Still, we'd be wise to keep in mind the potential benefits of a diversity of approaches and try to maintain some of the freedom and flexibility that characterizes so many ninth grade physics classrooms.

Continue reading →

Differentiated Instruction in Physics

Physics First provides opportunities for an interesting type of differentiated instruction, where students might arrive at a complex understanding of fundamental concepts through their exposure to a variety of experiences.

I just got back from a trip to Baltimore to observe Physics First classes. I had a chance to visit two of the fifteen Baltimore public schools now teaching CPO Physics First, but the details of this visit will have to wait until I have a chance to wrap my head around it somewhat!

In the meantime, though, I wanted to describe my experience at a monthly workshop for Baltimore physics teachers, called "Physics Works." Physics Works is a monthly gathering of physics teachers in the Baltimore public school system, and my trip happened to coincide with this month's meeting. The day's 2-hour workshop was based on planning differentiated instruction geared toward students' varied intelligences. (This idea was first suggested by Howard Gardner in 1983: humans exhibit multiple different types of intelligences: bodily-kinesthetic, interpersonal, musical, etc. An effective teacher can give students opportunities to participate in activities geared toward their personal strengths.) After discussing the varied strengths of the people in the room, we worked in small groups to complete the following task: develop a plan to teach Newton's laws that considers the multiple intelligences of students in a class.

Most groups in the room interpreted this task to mean providing students with a variety of assessments to choose from. These assessments could range from writing a physics rap to drafting a letter to Isaac Newton to analyzing of video footage from a football game. This is what the teacher running the workshop intended for us all to do, and she provided us with a handout on suggestions for creating a "menu" of options for students to choose from.

It took our group a while to notice this "menu" handout, and as a result our group interpreted the task to mean something very different. Instead of giving students a choice between assessment styles that might suit their strengths, we asked ourselves how we could exploit these intelligences to engage the students through different perspectives. We designed an activity on Newton's third law that began with students pushing each other outward in rolling chairs, to engage them through their bodies. We then imagined students breaking into smaller lab groups to discuss fictional students' interpretations of the results*, to engage them through words and conversation. Student might then work to design an experiment to collect quantitative data on mass and change in velocity with spring-loaded carts, to engage them through manipulating objects in space. This brought up a discussion of the link between logical-mathematical intelligence and spatial intelligence in data analysis using graphs and pictures, and so on.

What was exciting to me about this activity was that the concept of Newton's third law exists somewhere at the intersection of all these related activities. Students aren't just given the freedom to choose which types of activities they identify with more than others. Instead, they are expected to engage with the physics through each of the various activities, through areas of intelligence that they both excel at and struggle with. This seems similar to the multiple representations which lie at the heart of Modeling Instruction, discussed in the 11/17 post below.

When taken one-by-one, the "multiple intelligences" seem like a simplistic interpretation of human strengths and weaknesses, and indeed there is much to criticize in Gardner's theory. But perhaps the interaction between these intelligences is something to devote more attention to?



*These fictionalized discussions are used in Lillian McDermott and Peter Shaffer's Physics by Inquiry and Tutorials in Introductory Physics. These programs were developed by the Physics Education Group at the University of Washington, and they are both excellent. The format might look something like this for the activity discussed above:

Consider the following statements:

Student 1: A smaller student is always going to move faster because the larger student pushes them harder. Larger people are usually stronger than smaller people, so they have more pushing power.

Student 2: The recoil effect makes the person who did the pushing move backward, but this recoil never makes that person move faster than the person they pushed because their push is directed forward, not backward.

Student 3: Whenever two people interact, they push against each other exactly the same amount. A smaller person moves away from the interaction faster because they have less inertia, it’s easier for that same push to make them move.

Do you agree with any of these students? Explain.

Continue reading →