Getting it Right: Is Online Learning REALLY Better?
If you read a new Education Department study about online education and were not a professional researcher, it would be reasonable to come away thinking that on-line education helped students perform better than face-to-face instruction.
After all, the 93-page report says that the researchers “found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction.”
I found this to be such a startling conclusion that, if true, could spell the end of traditional education.
So I turned to professional education researchers at Columbia University to help me. When Shanna Jaggars, a senior research associate at Columbia’s Community College Research Center took a thorough look, a different picture emerged.
Here are some of the things she found:
*Though the authors conclude that on-line learning was better than face to face, they also present evidence suggesting that the better outcomes may not have been a result of the on-line media itself.
(For those who want the details: The report is called “Evaluation of Evidence-Based Practices in Online Learning / A Meta-Analysis and Review of Online Learning Studies,” conducted by SRI International. A “meta-analysis” is a review of a group of studies on the same subject. The authors found more than 1,000 studies conducted from 1996 to 2008 but settled on 51 that seemed complete enough to review. Most of the reviewed studies were done in colleges and adult continuing-education programs of numerous kinds. A minority involved K-12 education.)
*Within each reviewed study, on-line and face-to-face courses were often no equivalent in terms of instructional features. The observed positive effects may be due to more time on task, curriculum or other features of instruction.
*Thus the report does not support a conclusion that a significant shift from face-to-face to on-line courses would necessarily improve student outcomes. Instead, colleges should ensure that all courses, whether online or face-to-face, are of high quality.
*Researchers believe that students withdraw from on-line courses at higher rates than those in face-to-face classes. It is possible that the on-line courses retained the more motivated students.
*The authors in the report do not discuss whether the courses included in the reviewed studies are typical of most online courses.
Well, that all presents a somewhat different picture, doesn’t it?
I asked the co-authors of the study to respond.
Two of them, Marianne Bakia and Robert Murphy, said in an e-mail: “It is important to note that the meta-analysis does not support simply putting existing face to face courses online. It is not the Internet as a medium that is associated with student learning. If instruction is bad in a face to face context, simply putting it online won’t make it any better. However, the online platform does seem to support effective ways to redesign instruction and increase student time on task.”
They also said that while their work drew on the best available data, “unfortunately” the studies included in the analysis did not systematically address completion rates. And they said they “don’t have sufficient information” about the kinds of courses to know how representative they are.
The unanswered questions, then, remain.
This is not to say that the Ed Department meta-analysis is without value.
But this is to say that we all need to be careful about what we read and how much to read into conclusions.
New education studies come out every day. Some are revelatory. Some are garbage. I
It is important to see how the studies were conducted, who conducted them and how valid the conclusions really are. But the truth is that non-researchers (that means you and me) can’t always figure this out. We don’t know what we can’t see in an academic study.
It really is up to the research community to be clearer about the successes and the limitations of their work--and for those who see something that isn’t kosher to blow the whistle.
Let me know if you see a study or report that you wonder about and would like explained.
Every Tuesday in this space, we'll try to take a look at who's Getting it Right--and who's not.
Posted by: doglover6 | September 1, 2009 8:00 AM | Report abuse
Posted by: rogernebel | September 1, 2009 9:55 AM | Report abuse
Posted by: PLMichaelsArtist-at-Large | September 1, 2009 10:09 AM | Report abuse
Posted by: moreflowers | September 1, 2009 3:16 PM | Report abuse
The comments to this entry are closed.