Book Review: The Teaching Gap, by James Stigler

The U.S. has not done well in international comparisons in math. Our students score well below students in a number of other countries on TIMSS*, which tests students in 4th and 8th grade in dozens of countries. Stigler was part of the group of researchers who conducted an in-depth analysis of classroom videos associated with the 1995 TIMSS. They were looking for differences in classroom practice that would help to explain differences in scores. This book, published in 1999, is a fascinating description of their research results.

Classrooms in the U.S., Germany, and Japan were compared, and the main insight that came out of the study was that the culture of the classroom was very different in these 3 countries. Although it's an oversimplification, what they saw was something like this: In Germany, the teacher directs the students in developing advanced procedures, in Japan, the class works individually and in groups on structured problem solving, and in the U.S., the teacher leads the class in learning terms and practicing procedures (pages 25-46). One researcher said he had trouble "finding the mathematics" (page 26) in the videos of U.S. classrooms. (Yikes!)

Classroom culture is hard to change, according to Stigler, because much of it is deeply imprinted in us, as what school is. "The scripts for teaching in each country appear to rest on a relatively small and tacit set of core beliefs about the nature of the subject, about how students learn, and about the role that a teacher should play in the classroom." (page 87) Many teachers who've tried teaching with more of a problem-solving focus (including yours truly) can attest to how much resistance students put up: "That's not how math class is supposed to work! Just tell us how to do it!"

After viewing the Japanese lessons, a fourth-grade teacher decided to shift from his traditional approach to a more problem-solving approach such as we had seen on the videotapes. Instead of asking short-answer questions as he regularly did, he began his next lesson by presenting a problem and asking the students to spend ten minutes working on a solution. Although the teacher changed his behavior ... the students, not having seen the video or reflected upon their own participation, failed to respond as the students on the tape did. They played their traditional roles. They waited to be shown how to solve the problem. The lesson did not succeed. The students are part of the system. (page 99)

Which makes it clear that, however cool we think those Japanese classrooms are, we can't just bring their style over here as is. What we might be able to use here, however, is their lesson study process, modified to suit us. Teachers plan one lesson together in great depth, over a long period of time.

During lesson study, the teachers discussed what problem to start with, what materials to give students, what solutions and thoughts the students might come up with, what questions to ask, "how to use space on the chalkboard (Japanese teachers believe that organizing the chalkboard is a key ingredient to organizing students' thinking and understanding)", timing, working with different levels, and how to end the lesson. (from page 117, paraphrased)

Then they all watch in the classroom while one teacher plays out their plan with the kids. Afterward they all discuss some more, modify, and try it again in another teacher's class.

"Virtually every elementary and middle school in Japan is engaged in kounaikenshuu [lesson study]." (page 110) What Dan, Kate, and others are doing online (here and here, for example) might come close. Wouldn't it be great if we could start our own kounaikenshuu movement here?!


Here are some quotes I liked:

page 49:
He [Japanese math teacher] concludes by posting the goal for mathematics: "To learn to think logically while searching for new properties and relationships." He asks students to repeat this goal several times and memorize it.

page 75: (paraphrased)
The chalkboard as used as a visual aid that helps focus students' attention in the U.S. versus as a cumulative record of the day's lesson in Japan.
page 93: (regarding chalkboard use)
[Japan] Apparently, it is not as important for students to attend at each moment of the lesson as it is for them to be able to go back and think again about earlier events, and to see connections between the different parts of the lesson.

page 89:
Teachers were asked what was the "main thing" they wanted students to learn from the lesson. 61% of U.S. teachers described skills they wanted their students to learn. ... 73% of Japanese teachers wanted their students to think about things in a new way... to see new relationships between mathematical ideas.

page 90:
[In the U.S. view,] practice should be relatively error-free, with high levels of success at each point. Confusion and frustration ... should be minimized; they are signs that earlier material was not mastered.

page 91:
[In Japan] frustration and confusion are taken to be a natural part of the process, because each person must struggle with a situation or problem first in order to make sense of the information he or she hears later. Constructing connections between methods and problems is thought to require time to explore and invent, to make mistakes, to reflect, and to receive the needed information at an appropriate time.
...
Students will learn to understand the process [of adding unlike fractions] more fully, says the [Japanese teachers'] manual, if they are allowed to make this mistake [of adding denominators] and then examine the consequences.

page 94:
Japanese teachers view individual differences as a natural characteristic of a group. They view differences in the mathematics class as a resource for both students and teachers. Individual differences are beneficial for the class because they produce a range of ideas and solution methods that provide the material for students' discussion and reflection. The variety of alternative methods allows students to compare them and construct connections among them. It is believed that all students benefit from the variety of ideas generated by their peers. In addition, tailoring instruction to specific students is seen as unfairly limiting and as prejudging what students are capable of learning...

In Japan, classroom lessons hold a privileged place in the activities of the school. It would be exaggerating only a little to say they are sacred. They are treated much as we treat lectures in university courses or religious services in church. A great deal of attention is given to their development. They are planned as complete experiences - as stories with a beginning, a middle, and an end. Their meaning is found in the connections between the parts. If you stay for only the beginning, or leave before the end, you miss the point.

page 119: [Japanese teacher speaking]
Conceptually it's easy to break 6 down into 5 and 1, and it's easy to break 7 into 2 and 5, but it's really hard for first-grade students to break 7 down into 3 and 4. [!]

[During a lesson study meeeting] the teachers consulted some of the teachers' manuals and found 5 common ways of solving simple subtraction problems with borrowing.

page 127:
[Japanese] culture genuinely values what teachers know, learn, and invent, and has developed a system to take advantage of teachers' ideas: evaluating them, adapting them, accumulating them into a professional knowledge base, and sharing them.

-----
*The letters originally stood for Third International Mathematics and Science Study, which was conducted in 1995. At the National Center for Education Statistics website, the letters now stand for Trends in International Mathematics and Science Study, which is conducted every 4 years.

Tests .... (TIMSS & the MA teacher licensing test)

Part I. TIMSS.

Lately I've been reading a lot about the Trends in International Mathematics and Science Study, which has helped us compare the performance of U.S. students in math to that of students in other countries. It was given to students in 4th and 8th grades in 1995, 1999, 2003, and 2007, and the U.S. performs well below a number of countries, including Japan. Until just now I hadn't really thought to question the validity of the study.

But when I went to the NCES (National Center for Education Statistics) website, I saw Dare to Compare, where you can test yourself on the questions kids get. The very first question (I said I wanted 12th greade level) was on statistics, and required knowing the factoid that about 95% of all values in a normal distribution lie within two standard deviations of the mean. That's an arbitrary sort of thing, and not representative of important mathematical skills, in my opinion. [Added:] About 1/4 of the questions seem to be about probability and statistics.

Here's the question:

1. A clock manufacturer has found that the amount of time their clocks gain or lose per week is normally distributed with a mean of 0 minutes and a standard deviation of 0.5 minute, as shown above. In a random sample of 1,500 of their clocks, which of the following is closest to the expected number of clocks that would gain or lose more than 1 minute per week?

The answers provided are 15, 30, 50, 70, and 90. Before teaching statistics, I'd have estimated the area in the tails by sight, and guessed something less than 10% (150), and picked 90, which is wrong.

One bad question isn't enough to invalidate a whole test, but it does make me question the judgment of the people who created it. (Of course, it could be that the website questions aren't as carefully screened as the real test questions...) [Added:] Of course, if all 12th graders are expected to have taken statistics, then there's not much problem. Is that the case these days?


Part II. MA Teacher Licensing Test

Massachusetts recently instituted a math test required for teachers to get licensed. Only 27% of the people taking this test passed it. [The score needed to pass is not available, as far as I can see.] Are the candidates unprepared mathematically? Most likely some of them are. Is the test reasonable? Well, you can get it here, and judge for yourself. [Note: A news article originally led me to the wrong test, what looks like a math subject test for the high school teachers. Thanks, Colleen for pointing out the right one. Both are on this page.]

The test has 45 multiple choice questions and an essay question, in which the test-taker is asked to evaluate a student's work. I took the test just now, and got a few wrong. (Achoo! My brain doesn't work today, with this cold.) About 2/3 of the questions should be pretty straightforward for someone well-prepared. Then there's 12 to 15 questions that will really take some thinking. Is that too much to ask in the stressful situation of a licensing test?

[Commentary on wrong test deleted.]

Here's a quote from a Boston Globe article:

"The failure rate isn't because we have placed some arcane, really hard, high-level math on the test," said Robert Bickerton, associate commissioner for the Elementary and Secondary Education Department, "What we've done is say you need to know this math deeply and you need to be able to answer your students when they ask why."

Hmm, I wonder if he's taken the test. I do want math teachers to be able to think deeply, but this test may be asking a lot. I guess I'd make it a bit shorter, and maybe put the more straightforward questions first. What do you think?

[Burt pointed out some great articles, but his links didn't work. I don't know how to do links in comments, so I'm adding them here: Patricia Kenschaft at 2005 AMS meeting, Racial Equity Requires Teaching Elementary School Teachers More Mathematics, and Deborah Ball et al, Knowing Mathematics for Teaching]