Education is a complex and messy endeavor. We all have our own ideas about how children should be raised, about how learning happens, and about what is important for children to learn. The schools have to deal with parents on all sides of every political spectrum demanding what they think is needed for their children.

Educational research that doesn't acknowledge this messiness, that tries to buy 'scientific' cachet with control and treatment groups but frames the questions too narrowly, is more likely to reinforce the values of one group than to deepen our understanding of the learning and teaching enterprise. (Of course, we're each more likely to see flaws like this in research that doesn't resonate with our own values.) In this post, I want to dissect a flawed study, together with my friend Ben, and then give links to some studies I've enjoyed reading. I would actually appreciate it if any of you would like to point out flaws in some of those. (I'll start a new post for each one, so we don't get all tangled up.)

Thumbs Down

“The Advantage of Abstract Examples in Learning Math”, a study done by Jennifer A. Kaminski, Vladimir M. Sloutsky, and Andrew F. Heckler, was published in Science magazine in April 2008 (you need a paid subscription to see the article) and highlighted in the New York Times soon after. I had responded to this study before I started blogging, in email to colleagues. It was brought to my attention yesterday by Ben Blum-Smith's new blog, Research in Practice, where he writes a great critique of it. I agree with all he says (here and here) and want to add a bit more.

The people who did the research, along with the unquestioning NYT author, say that the research shows that students learn math better with abstract examples than with concrete examples. Ben and I are saying that their research design is severely flawed, and that they've shown nothing useful.

Ben gives a great description of what the research folks were supposedly trying to teach, which was the properties of a "commutative mathematical group of order three". That's a fancy name for something not much more complicated than clock arithmetic. Imagine a clock that only has three hours on it, and instead of a 3 at the top, it has a 0. So 1+1 is still 2, but 1+2=0 and 2+2=1. This is the most common example used when people are first learning about these groups. The examples used in the research seem very contrived in comparison.

Both the research folks and Ben described a tennis ball factory, where you're keeping track of how many balls you have in hand, after you've put as many as possible into those 3-ball cans, but Ben's description makes a lot more sense than the one used in the research. (When I originally read this study over a year ago, I never saw the tennis ball example. In the one 'concrete' example I was able to find details for, they used a full cup for the 0, or identity element, which I would find confusing.)

People who actually study groups like this sometimes do it without numbers. The 'elements' of the group might be labeled a, b, c instead of 0, 1, 2. There are properties that can be studied, like identity elements and inverses. (0 is the identity because adding it to other elements doesn't change them. 1 and 2 are inverses because 1+2=0, the identity. These properties can make sense even when the elements aren't numbers.) So the researchers 'taught' this using 'abstract' examples for some subjects and 'concrete' examples for others. They quizzed all of the subjects using a group consisting of a vase, a ladybug, and a ring. Although concrete, these strange elements fit much better with the 'abstract' example than with the 'concrete' example. It's not surprising that the subjects whose example was more similar did better when quizzed.

There is lots of narrowly focused research like this out there. It may be useful in physics to narrow a question down to one detail, when the interactions between the small parts is clear. But in social arenas, all the parts interact, in very complex ways. Research like this cannot tell us much of value, even when its design is less flawed.

Kaminski et al want to say that their research tells us children learn math better without concrete examples. Their claim is very political. To promote it, they have done a number of studies with minor variations. (Googling their names, I see work done in 2003, 2006, and 2008.) I'd rather see education research that addresses the big, messy picture. Here's the MAA president-elect's take on this, and here is an article interviewing one of the 3 authors of the study.

Thumbs Up

Research that I've found more interesting is much broader. It doesn't attempt a double-blind statistical power, which can only come through narrowing the questions until they become too artificial to be of use.

I've been reading Jo Boaler's work on the benefits to students at all skill levels of working together with one another. On first glance, it may seem that tracking would allow the best students to go farther, and allow slower students to get a more solid grasp on what they're studying, but tracking actually harmful to students at both ends and those in between. Boaler shows why, and shows how to make heterogeneous grouping work. One article is here, or you can read her book, What's Math Got to Do with It?

Alan Schoenfeld wrote the book Mathematical Problem Solving, in which he describes his very detailed research into the process followed by math students versus mathematicians while attempting solution of a hard problem. He taught a course in problem-solving strategies, using Polya's framework, and gave a pre-test and post-test to those students. Compared to students taking a more typical math course, these students' problem-solving skills improved significantly more. Here is an article of his on a different topic, how mathematical conversation in the classroom promotes learning.

Both Boaler and Schoenfeld compare groups with and without the 'treatment' they believe is effective, and show evidence for their belief, as Kaminski et al did. There are other sorts of research, which attempt to understand children's learning processes, but which don't have 'control groups'. One such project I'm interested in following is Measure Up, which introduces math through measurement and algebraic reasoning. Here's an article from that project.

Our biggest problem may not be understanding better how children learn, but implementing the good ideas that come from this research. When most elementary teachers are uncomfortable with mathematics, what we need to focus on is how to help them. Liping Ma's book, Knowing and Teaching Elementary Mathematics, compares elementary teachers in the U.S. and China. The Chinese teachers understand the math much more deeply. A good summary of the book is here. And an article by Ma is here. And here's a piece by another researcher, Hung Hsi Wu, on the depth of understanding needed by elementary teachers.

What math education research have you found valuable?

## Friday, October 16, 2009

Subscribe to:
Post Comments (Atom)

none of it. it's *all* about who gets funded

ReplyDeleteand it's useless by design. one has understood

for generations how to get the actual job done.

coming up with better and better ways to lie

about it demeans mathematics, learning,

and life itself. maybe i'm just in a bad mood

but i feel this way more often than not so

there it is.

I have to say that I work hard at teaching, and I don't know 'how to get the job done'. In my opinion, the best learning happens when it comes from within - when the student comes to the topic through their own passion.

ReplyDeleteBut in school classrooms, we try to get the topic to the students on some schedule. Are there good ways to do this?

If most people in the U.S. are uncomfortable with math, I think there's a serious problem that needs solving. I don't know the solution, and reading these studies helps me think about it.

How do you think we ought to go about getting the job done?

And thank you for a thought-provoking reply! :^)

ReplyDeleteyou get it done... as you know very well...

ReplyDeleteby listening to students not by listening

to bosses. art vs. commerce. no mystery.

also:

people not understanding math

is a problem for some *and*

an opportunity for (many) others.

as long as *we* play

"what can we do for you?"

the teams playing

"what can you do for us?"

will gladly take our lifetimes

of service... but they

won't change the rules of the game

just because it's the right thing to do:

this is *politics* for god sake.

polite discourse forbids mentioning this.

also race and class and money

and all the things you have to talk about

to say anything meaningful about "education".

one is trying to save the world.

too big a project. when the point

is to help the person in front of me

there are usually all *kinds* of

available useful things to try

and everybody knows this.

how is everybody in this big useless

disappointment machine gonna get paid though.

also: passion is easily stimulated

(or even simulated). *devotion*

is what gets the payoffs.

and *only* the already-well-off

can *afford* to be devoted to

"learning for its own sake".

this is also very well known.

and impolitic to mention.

Hey Owen, I wasn't sure it was you in the first comment (anonymous, you know). Good points all, and not at all impolitic to mention here.

ReplyDeleteHow do I listen to 40 students at once? How do I deal with the travesty that is grading?

I *did* get something out of Boaler. She gave me evidence that group work can be really valuable, that tracking is damaging, and that it's worth my time to try to learn how to manage groupwork better.

I often wonder whether trying to teach problem-solving skills is useful. Schoenfeld gave me evidence that it is.

I keep thinking about Devlin's thing about 'multiplication is *not* repeated addition'. I'm not sure that I agree about the importance of this. But the Russian stuff about starting out with measurement seems interesting, so I'm doing some of that with the kids at Wildcat.

I do think all of the research I mentioned above (thumbs up) has had some effect on my practice. (Except the last, about elementary teachers, but if we get the grant that would have me teaching elementary teachers, then that will probably affect my practice also.)

"Useless by design" goes too far, imo, but I'm interested in your take on the politics of research funding.

Thanks for voicing your opinions. I value them even (especially?) when I don't agree.

Hey Sue, thanks for letting me know about Schoenfeld and Boaler. I love Liping Ma's book. Did you want to make some plan for reading Wu's article together?

ReplyDeleteWanna discuss it next Friday? (Any time's ok with me. I've finished it.)

ReplyDeleteI think you'd like Cognitively Guided Instruction (Carpenter and Fennema and someone else). They've done a nice book: Children's Mathematics (Heinneman) that's quite readable. (Low grade level)

ReplyDeleteThanks for the lead.

ReplyDelete