Why teaching defies measurement

Campbell's Law, formulated by Donald T. Campbell in 1976, states "The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." This is closely related to Goodhart's Law and the Lucas Critique in economics and a lot like the Heisenberg Uncertainty in quantum physics which says that the act of measuring something changes the thing being measured.

For example, the recent scandal coming out of D.C. that a disproportionate number of erasures on answer sheets might explain the relatively huge jump in scores on mandated assessment tests seems to follow Campbell's Law. If a school or, in the case of D.C., a district commissioner knows that money and reputation are tied to performance, there will be efforts to cheat. Furthermore, the act of testing (measuring) will change the thing being measured. Mandated tests don't measure actual schools; they only measure how schools are when being measured. There is a difference.

Think about this when you read The Denver Post's editorial today (Friday, April 15, 2011), "Stay the course on school reform." This is an inevitable follow-up of the Posts' long and loud endorsement of SB191, Colorado's "landmark" teacher evaluation legislation. You know, that was the bill that changed the way teachers were to be evaluated (half of their evaluation will be based on standardized test scores) and attempted to put an end to tenure. The thinking behind the bill was and continues to be the current view that if we just got rid of all the incompetent teachers - the ones who are in the profession only for the money - all our educational problems would be solved.

Today's editorial is championing a state panel's suggested guidelines for teacher effectiveness and encouraging us all to support the next step, a compilation of quantifiable objectives and behaviors teachers would have to follow on the road to the retention of tenure.

The state board came up with 60 recommendations and as the Post says, "those recommendations comprise an important foundational document for the work ahead." The Post goes on to highlight a few of these important and presumably new recommendations. For instance, the board wants all teachers to guarantee "equitable learning opportunities and growth for all students." The board also recommends that good principals will be "responsible for the collective success of their schools."

"That sliver of the panel's work provides an idea of the broader principles," the Post remarks.

Wow! "Equitable learning opportunities." "Growth for all students." Why didn't I think of that? Just think, 35 years of teaching down the drain. If only there had been some government board to tell me how to proceed. "You mean I'm supposed to be concerned for ALL of my students? Well, that takes all the fun out of it right there."

The Post strongly suggests that all this new thinking and the new battery of ways for hapless principals to quantify everything is going to change the way educators think.

This argument is a good example of the kind of thinking that ensues when people who are on the periphery of classrooms try to turn teaching into something that can be measured.

In the three and a half decades I taught I was forced to sit down with the rest of the faculty at least once every three years to arrive at new methods for improving instruction. Each one of these sitdowns started out with a listing of goals and all of those goals sounded a lot like those "Equitable learning opportunities" alluded to above. We agreed to prepare students for an increasingly complex society. We agreed students should be able to meet the needs of society by effectively expressing themselves. We agreed they should have a basic understanding of government, etc., etc.

The goals are self-evident and nothing to cheer about. The problem comes when we have to measure our success.

Interlude: I had a kid once who bombed his CSAPS because the night before his mother and her boyfriend, who combined to lay the foundation for the kid's cocaine addiction, were arrested at his home. He snuck out the back window and roamed the streets all night. To make matters, the little creep did poorly on his reading test which is probably why I didn't get that great evaluation I was counting on.

There are too many variables. That's why teaching is an art. Would anyone seriously want to come up with a list of 60 characteristics of an effective artist? Characteristic 16: Has fondness for absinthe and self-mutilation.

My wife informs me that principals in DPS have a checklist of over 100 items they must use when evaluating teachers. To DPS's credit or damnation, the items are specific and measurable. Instead of something like "the teacher will conduct large group discussions during each unit," it will be more like "during a large group discussion the teacher will demonstrate effectiveness by having 70% of students with hands raised."

I have heard that many DPS principals are considering early retirement rather than go through the check list charade. I can't imagine how a principal could pull it off. When I taught I was lucky if an administrator was in my classroom more than twice a semester. The demands placed on principals are just too great. How many times would a principal have to visit a classroom in order to check off 100 behaviors? The whole notion is laughable.

The Post ends its editorial. "There is a long road ahead before a new way of looking at teachers and principals is fully deployed in Colorado. We hope the ensuing steps are as sound and purposeful as this one."

Trust me. If history and my experience are any indication, they will be every bit as "sound and purposeful." That's precisely why I can't stop rolling my eyes.