Why School Reform Fails: A Case Study

A valuable article in Sunday's Strib about Q Comp, Governor Tim Pawlenty's "performance pay" system for teachers. The article notes that 99% of teachers in districts that participate in the Q Comp program receive the financial incentive it awards. It also notes that the districts participating in the program have shown no discernible improvement in their achievement levels due to the system. There is much we can learn from this simple tale of reforming a-go-go.

A GOOD IDEA . . .
Q Comp was supposed to reward highly effective teachers. Who doesn't like the sound of that? The idea was to begin moving away from the current system, which rewards teachers based on two criteria--their own educational achievement (coursework and degrees) and experience (number of years teaching). Unfortunately, these two criteria haves no correlation to the results toward which we want teachers to strive like quality instruction or student achievement. Creating a system that does reward these results, if only partially, is one of many areas reformers seek to change in our public education system.

RUN THROUGH THE POLITICAL PROCESS . . .
Even the Governor cannot simply dictate such changes. Public education is, after all, still public, which means that systemic changes are run through the sausage factory of legislators, school boards, unions and bureaucracies before being enacted. Many times, such as this one, the results of the factory no longer resemble the original intentions. While Q Comp started off as a way to reward highly effective teachers, it became an enhanced version of the compensation system we already have in place. Teachers are rewarded in Q Comp for taking more classes, sitting through more professional development, discussing more books and generally spending more time focusing on things that ultimately do not improve the quality of education, well-intentioned they may be.

DOESN'T CREATE CHANGE . . .
The culture of education permeates more than just schools. What I mean by this is that public schools believe nearly every challenge we face can be solved by . . . more education. Professional Development is a way of life for teachers and school districts, even if the results are non-existent. When faced with poor performing schools, teachers or principals, the first reaction is always to provide more training. (Whether or not training is the problem.) Q Comp, as it currently exists, helps feed this culture of focusing on inputs and training rather than on outputs and results. The system rewards performance--but essentially that performance is jumping through additional professional development hoops, not changing behavior.

OR THE INTENDED RESULTS . . .
Needless to say, it should not be a surprise that Q Comp is failing to have any impact on student achievement. Even if Q Comp did, in fact, reward those teachers who demonstrated great practice and high quality instruction, it is pretty doubtful that a single change to the compensation system alone would have a meaningful impact on student results.

THEREBY PROVING REFORMS DON'T WORK.
See? What did all this hard work and reform get us? Nothing. Teachers unions can now say with confidence and well documented data that performance pay doesn't work. Conservatives and anti-school folks can look at Q Comp and show how we're pouring even more money into schools without getting anything in return. Reformer-types can claim the program was so water-downed as to be worthless. School districts can blame the state for creating a new program that didn't do what it was supposed to.

Meanwhile back in classroom after classroom . . . .

Spotting Teacher Talent

So I posted a recent New Yorker article over on my facebook page that I thought was quite excellent. I thought this because A) Malcolm Gladwell creatively and seamlessly connects two apparently unrelated topics, namely predicting the success of a college quarterback in the NFL and predicting the success of a classroom teacher and B) because I agree with his conclusions. Turns out this has generated much discussion! Four Five comments at last check! Exciting.

So a few quick comments about the article. One of the points Gladwell makes is that we don't know who a successful teacher will be until they are actually teaching in a classroom. Very much like trying to predict which college quarterbacks are going to be successful in the NFL, there simply is no clear set of predictors that allow us to say with confidence that a person will become a good teacher. The reason this is important is we often try to address teacher quality by doing things that sound like they should help teachers, but in fact have nothing to do with successful teaching.

Currently the only tangible rewards that we offer teachers are based on their experience and educational credits. A teacher gets paid more and moves up the seniority ranks by taking more courses and teaching more years. The problem with this is that research shows us it has absolutely nothing to do with what the teacher actually does in the classroom. A bad teacher with a masters degree is still a bad teacher. A great teacher with three years of experience and a lowly BA is still a great teacher. There is no correlation between asking teaching to take more classes and the results we will see in our schools. That was the point I was trying to make in my little facebook comment.

Michael read the quote (and likely the whole article) and said that, "It would be nice if there were some more reliable marker than just students' test scores." (In all honesty, I am not entirely sure if Michael was referring to the fact that it is too bad we don't have a better marker to indicate who will be a successful teacher OR if it's too bad that we determine a successful teacher mostly by test scores. I assume it was the latter.) Which opened up the conversation about testing. Let me say briefly that I understand the point that tests alone do not indicate teacher quality. To be sure, standardized tests are blunt instruments that tell us one piece of the puzzle. I would, however, argue that they represent a pretty big piece of the puzzle.

Researchers who study school test data can clearly identify the impact that teachers have on their students test results while controlling for race and poverty. Year after year we will see that certain teacher's students continue to improve at a faster rate than average and certain teacher's students fall behind. This should make sense, right? If we believe that teachers are important to the education a student receives (as nearly everyone says) than we would expect to see that better teachers create better results. I would argue that being a loving, caring supportive teacher should not overcome the fact that students are falling academically behind after spending a year in that teacher's classroom. It would be nice to have complete, holistic evaluations of teachers (and by the way, we do know how to do that; we haven't figured out the executional and political challenges of making it happen in most public school districts), but until we're there testing data seems a pretty objective, valuable tool to begin the conversation. Much more on testing in the months to come, I'm sure.

Miscellaneous: School Stuff Edition

.: The Strib has been putting out a slew of interesting education articles in the last week or so. This piece explores teacher evaluations at Saint Paul schools that are NCLB-required to restructure. Note as well the boilerplate discussion in the comments section.

.: A significant report on charter schools in the Twin Cities was released just prior to Thanksgiving by the University of Minnesota's Institute on Race and Poverty. Here's the Strib's take and MPR's. (Summary: some work; some don't.) Several interesting data points to look at here, but let me point you to this chart which shows the correlation between reading proficiency and poverty. You might think there's nothing new to see there and, in general you're right. But there is something very important indicated by these data. Look at the data points above the black line. When you see numbers in the 70s and even 80s, that's not random chance. (Same chart for Math results here.) That's effective schools and effective teachers improving the results that they deliver for their students. Many people look at the correlation between poverty and educational performance and assume that there is no possible way to ever expect schools with low-income kids to show improvement. The data clearly show that's not true. Great teachers and great schools can teach students from the most challenging backgrounds. Too many people (including, sadly, some in the schools) don't believe that.

.: How Washington County schools are seeking grants to support their World Cultures Day.

.: Unhappy parents in Eden Prairie.

.: More Michelle Rhee.

.: I want to acknolwedge the fact that I'm not really providing much analysis or insight yet into my posts on school reform. The main reason for this is that I'm still absorbing, still learning. I've got a couple of half-baked opinions already about several things, but I don't have enough experience yet to make the case. So if it seems like I'm just throwing up bland articles to read or non-controversial fluff, I apologize.

Getting Great People in Schools

So I went to my first educational conference last week. On the whole a positive and inspiring experience. The keynote speakers were a who's who of educational reformers (or, depending on your point of view, loud-mouth-know-it-all's-trying-to-foist-their-unproven-ideas-on-hard-working-school-employees-everywhere.) The focus of this particular group was managing the people side of schools. It wasn't about curriculum or educational theory, but rather about what it takes to attract, retain and support talented folks who want to make a career in our public schools.

Many organizations, and schools in particular, struggle to align the various aspects of how they manage people with their overall objectives. If we want to get a great teacher in every classroom and a great principal in every building, than wouldn't it be cool if our hiring, retention and developmental practices align with that fundamental purpose? In fact, how we do all of those things currently is often at odds with creating that quality. In broad brush strokes, here is the current state of affairs for how we manage people in most public school districts:

Budget timing and dollars are completely separate from the recruiting and academic functions. Schools often don't know how much money they will have until late in the hiring cycle.
Recruiting is done at a blanket, district level where the evaluation criteria is nearly all credentials based rather than quality based. Due to budget timing noted above, many schools start the school year without proper staff and then scramble to find someone, anyone with a license, to fill gaps in their staffing.
Retention allows little opportunity for quality to be evaluated. In the case of teachers, school districts do a poor job of managing tenure and, once tenured, there is little opportunity to improve performance.
Promotion essentially doesn't exist. There are few rewards (rank, financial, responsibilities or otherwise) to offer individuals who are demonstrating great quality. A great teacher may choose to move on to become an administrator, which, while financially beneficial to the teacher, may be a net loss for the district as it removes talent from the classroom.
Professional Development is often driven by vendors/third-party providers with little alignment to overall goals and little rigor. We perpetuate a culture that says: by taking a class, you will get better at your job.
Salaries reward longevity and credentials--which have no correlation to quality. On the whole, salaries are generally low in comparison to other career options. (As a sidebar, it should be noted that there are many examples and studies showing that salaries are not the sole barrier to attracting highly talented individuals to schools. Many would consider a career in teaching with the existing salary rates. Yes, it's important to see salaries rise, but that is by no means the only or even most important action required to attracting talent to our schools.)
Staffing of individual schools is usually based on seniority. The foundation of our staffing decisions is not matching skills and talents with the needs of a building (a great suburban principal may be a disaster in the inner city and vice versa) but instead is based on seniority or administrative assignment.

And on the list goes. What we say is that we want to create a school system dedicated to getting a great teacher in every classroom. What we do is quite different. It's important to note as well that these problems are a shared responsibility. Some of them have to do with rigid contracts, but many of them have to do with administrative management as well. An integrated, comprehesive approach to attracting talent requires a comprehensive overhaul of how we do our work.

A Large Convergence of Challenges

At the very first school board meeting I attended, the superintendent began listing off the overwhelming challenges we face. He started to say it was a "Perfect Storm" but quickly corrected himself and said instead that we face, "A Large Convergence of Challenges." Indeed.

After making the transition from Corporate America to Public Education four weeks ago now, there are several things that have surprised me. Oh sure there's the slow moving bureaucracy, the culture of low expectations and an office building that looks like it belongs in Detroit. All of that was completely expected. But what's been interesting are the things I didn't expect.

Perhaps the most interesting eye-opener for me so far is the interdependency of the issues facing urban schools. Nothing exists as a discrete problem and rarely are there resources available to fix one problem without creating, or at least impacting, another. Take the foundation of a school district--the school buildings. Suppose you have a school building that isn't fully used to capacity. It was built 50 years ago when the district had tons of students, but today it's half or three-quarters empty. Perhaps it's in a run down neighborhood or has a bad academic reputation.

Being a good steward of our public resources, one might try to find a way to use this facility at 100% capacity. After all, that's when the building is most cost-effective. Besides, there are always programs and other schools in need of space.

So your options might go something like this: you could move some of the special programs (like English as a second language or Special Education) into your semi-empty building. Doing so would likely bring down the overall academic profile of the school since these populations tend to have lower achievement rates. Doing so might also affect the overall racial mix of the school, adversely affecting the ability to maintain desegregated schools. Some people might say that you were trying to remove low-performing students from a 'good school' in order to shuffle them off into a 'bad school.'

Another option might be to keep those special programs where they are, but move kids from crowded regular programs into your semi-empty building. The challenge there is that's pretty unlikely that your crowded buildings are next door to your non-crowded buildings. In fact, they may be on the opposite sides of town. So by doing this you will either disrupt hundreds of students by re-drawing attendance districts and forcing students into different schools OR you will be busing students across town for 20 or 30 minutes one way. In fact, nearly everything you do will have a transportation cost associated with it (which already takes up 30% of the district's budget). And none of this touches on the political issues that parents and school board members will likely raise when any program moves anywhere.

The point of this is to simply demonstrate that nearly all of the various desirable goals to which we hold schools accountable are constantly in conflict. While it seems pretty straightforward to say that our decisions should be focused on doing whatever we need to do to improve academic achievement, doing so in nearly every case is going to have an adverse affect on some other metric that we value--segregation, cost, parental choice, neighborhood schools, efficient use of resources, etc. These aren't impossible decisions. They aren't insurmountable problems. But they do require courage and leadership that can be in very short supply.