Gates Foundation study: We’ve figured out what makes a good teacher

It's common for people who received good educations to say that test scores aren't an acceptable indication of learning. It's probably true that if you already have an acceptable level of education, test scores shouldn't be too important, but it seems like the people criticizing test scores haven't experienced what education is like for the children being "left behind."<p>In middle school, the last three weeks of each semester were spent watching movies (the same two or three movies over and over). In high school, I was taught that the big bang theory was when the continents divided. I was taught that the most direct route between two points on the globe is through the north pole. My Spanish and woodworking teachers taught math because they knew that the math teachers weren't doing it.<p>When you're talking about schools like the ones I went to (which weren't nearly as bad as many other schools in America) I think that test scores are a decent indicator of the education a student is receiving. You're measuring things like "can this kid read English" and "can they multiply two numbers together." Standardized tests are certainly capable of measuring things like that. Testing probably breaks down for measuring intelligence/education as the level of education improves, but I encourage everyone to consider just how dysfunctional many schools are before ignoring testing entirely.

I'll have to look at the prior reports, but this 3rd report is somewhat disappointing in terms of its statistical rigor.<p>They do provide weightings based on optimal predictions for maximum accuracy with predictions for state tests scores, although those weights appear to assume a simple linear correlation. Even then, the data suggests that in class observations are almost a complete waste of time (single digit weightings), and that student surveys decline in value as the kids mature (no shocker: with younger kids, how much they like the teacher has a much more profound impact on their success at learning). In all cases the kids seem to be a more reliable indicator than the observations of professionals! This may just be an indication that assessments based on more frequent observations are more accurate, but the only way we can really get a sense of that is if they also had peer observations from team teaching environments... which are completely absent.<p>That's not the worst of it though. The worst is the rest of the data. It is based on some fairly arbitrary presupposed models for combining these different assessments. No breakdown of the individual components of these assessments (other than "English" &#38; "Math") and no attempt to discern a model from first principles.<p>There are some other factors as well. Aside from some outliers, the accuracy of predictions seems kind of suspect. Most of the outcomes are within +/-0.05 standard deviations, and a majority of them are within what looks like +/-0.025 standard deviations. These are pretty minuscule variances (0.25 was supposed to represent a year of schooling, so 0.025 represents maybe 15 school days worth of progress at best), and it is not hard to see that without some of those outliers, the regression line doesn't look so pretty (particularly for English). I also didn't see any metrics on the variance of the data within a classroom, which I suspect makes any predictive value look limited at best.<p>Ugh. I really hoped that Gates would make sure there was more statistical rigor with an analysis like this. :-( I hate to say it, but the conclusions of the study are not well supported by at least what is in the final research report.

I can't say whether the findings are correct or not, but I like this:<p>&#62;&#62; The teachers who seemed to be effective were, in fact, able to repeat those successes with different students in different years, the researchers found. Their students not only scored well on standardized exams but <i></i>also were able to handle more complicated tests of their conceptual math knowledge and reading and writing abilities.<i></i><p>To everyone who keeps saying "standardized tests are bad", please cut it out. A <i>particular</i> code benchmark may be useless, but benchmarking overall is good. <i>Existing</i> standardized tests may be bad, but in general, having some way to make objective measurements and adjust accordingly is the basis of all human improvement.<p>And yes, we should reward good performance, if we can determine what good performance is. How else do you expect to get more of it? It <i></i>is<i></i> incredibly difficult to make incentives that actually do what you want rather than encourage gaming of the system, but that doesn't mean we shouldn't try.<p>In short, let's stop debating <i>whether</i> we should measure and incentivize, and start debating <i>how</i>.

There are many important factors which teachers (no matter how good they are) can't control:<p>1. How wealthy, poor, educated, or uneducated a child's parents are.<p>2. How many resources the school and teacher are given.<p>3. Whether a child's parents are abusing them at home.<p>4. Whether a child is witnessing violence, hopelessness, despair in their neighborhood.<p>5. The attitude of a child's family and friends towards education.<p>6. Whether a child's parents are active in their child's education and in a PTA.<p>Study after study has shown that these factors greatly influence a child's attention and attitude to school work, their attendance, and ultimately their performance at school.<p>No child's performance on a standardized test, and no evaluation of a teacher's performance is going to pick up on the above factors, yet they can critically impact how well the child ultimately learns and what the child learns.<p>Standardized testing of a child that is unfortunate enough to live in a situation where the above factors are against them will likely notice a poorly performing student, one who gradually does worse and worse on the tests until they finally flunk out.<p>Then, instead of getting extra aid, the funding to the school that has a lot of such children will be cut (thanks to programs like No Child Left Behind), and their teachers penalized.<p>These factors, which are strongly associated with poverty, are things a given teacher is in no position to influence, but which society as a whole could address. If it has the will.

I, like many, hope the MET project yields fruit.<p>However, see this post for a strong critique of one of the key claims of the MET study:<p><a href="http://garyrubinstein.teachforus.org/2013/01/09/the-50-million-dollar-lie/" rel="nofollow">http://garyrubinstein.teachforus.org/2013/01/09/the-50-milli...</a><p>Namely, in the scatterplot correlating predicted effectiveness and actual effectiveness, they pooled teachers into 5% groups. This naturally reduces the variance, yielding a much straighter line of points than would an ordinary scatter plot. This is not, to my knowledge, an accepted or ordinary technique.

Actually they have figured out how to evaluate teachers based on statistical models based on their repeatability of success. Finding a 'great' teacher is much more than finding the top rated teacher in my opinion. What qualities makes such a teacher good is a totally different matter. The latter was what I was hoping to find.

"We’ve figured out what makes a good teacher" They've figured out that a "good" teacher can repeat the process of success (whatever this is), there is no information how he made it. The headline is slightly misleading.

They haven't figured out a damn thing regarding what makes a good teacher. At best, they've figured out how to find which teachers are good. That's not nearly as helpful. It's barely helpful.<p>What happens when you find the bad teachers? Fire them and hope the fresh-faced college grads do a better job?<p>So much of this debate on teachers is leans towards the carrot or the stick. Wanting to reward the best teachers because it makes you feel like you're helping is fine, but don't fool yourself into thinking it'll hugely improve instruction. Teachers aren't going to suddenly go, "Oh, wait, you'll pay me more if I'm a good teacher? I'm going to start trying!" (One or two might, but those outliers exist in any field, and are hardly the issue.)<p>In every district I've taught, very little time is given for teachers of the same subject to collaborate, observe each other, and improve their own teaching. Every year, we have to get 20 hours of professional development. Tell me, are 6 minutes and 20 seconds of development per teaching day going to improve instruction?

The article kindly submitted here links to the project homepage<p><a href="http://www.metproject.org/" rel="nofollow">http://www.metproject.org/</a><p>at which the latest project report can be found. I've got some reading to do before I think about what else to add to the discussion already underway here. I get the impression that one observation that has motivated much of this research is how persons educated overseas have fared in United States society after immigrating here in early adulthood. My own experience living in another country (twice, in two different three-year stays) does much to prompt my interest in education reform, which is what drew me here to Hacker News.<p><a href="http://news.ycombinator.com/item?id=4728123" rel="nofollow">http://news.ycombinator.com/item?id=4728123</a>

If this becomes policy, it will most likely become another example of Goodhart's law. <a href="http://en.wikipedia.org/wiki/Goodharts_law" rel="nofollow">http://en.wikipedia.org/wiki/Goodharts_law</a><p>You can find a measure that correlates to performance, but once you use it to actually evaluate teacher, teacher will adapt and the measure will lost all relevance.<p>Rely on test scores? Teach the test, not the material. Rely on student evaluations? Bribe them with easy material and good grades. Rely on principal evaluation? Get on its good side.

This work is valuable, but I personally would like to put more focus on <i>improving</i> teaching and learning - helping teachers do better, helping students do better, creating better learning environments and curricula.<p>For example the budget of this study came out to about $15000 per teacher. What if instead we spent $5000 of that for professional development / training for the teacher, $5000 for a classroom set of chromebooks, and gave the teacher a $5000 raise. That alone would cause some significant improvements, I believe (not that that is ALL that is needed, and there are so many bigger issues that need to be tackled as someone else mentioned, such as poverty and inequity).<p>But when we talk about "good" teaching and "bad" teaching, we are treating it like it is a simple craft that is mastered or not mastered. Teaching is like engineering, it involves design, and design is about making things better (see this prezi on learning design: <a href="http://prezi.com/b44jwdgvs8nl/olds-mooc-introduction/" rel="nofollow">http://prezi.com/b44jwdgvs8nl/olds-mooc-introduction/</a> )

The study identifies teachers who are effective against a particular metric. Most identified are likely members of the subset of good teachers. Yet, the three R's as measured by standardized test outcomes are not the only measure of effective education, just one that is cheap to measure.<p>Dewey believed that the goal of education should be to produce good citizens. Systems with metrics based upon the model of student as consumer are more likely to produce outcomes biased toward producing consumers.<p>The entire set of good teachers includes those who inspire a lifetime of learning. It includes those who teach kindness, sportsmanship, and good humor. It includes those who teach subjects for which there are few students and in which standardized tests make little sense such as art and shop.<p>Maybe the study can point us in the right direction. But few districts will bring outside evaluators from Cambridge's famed institutions. Few districts will fund ETS analysis or purchase their tools.<p>What we are likely to get are processes localized to the political reality on the ground.

Evaluating teachers is all well and good, but the key to improving our educational system is not that. What is needed is <i>improving</i> teachers. See "Building a Better Teacher"[0]<p>[0]<a href="http://www.nytimes.com/2010/03/07/magazine/07Teachers-t.html?ref=magazine&#38;pagewanted=all&#38;_r=0" rel="nofollow">http://www.nytimes.com/2010/03/07/magazine/07Teachers-t.html...</a>

Is this the same gates foundation that I saw on 60 minutes pumping GMO seeds and fossil fuel fertilizer to third world countries to "help" them out of their food crisis?<p>In fairness, I think I heard they realized something like "what works in america doesn't work everywhere" (bit of a cop out to save face) and moved toward more sustainable organic methods, but I'm not completely sure.<p>All I know is, if its that hard for them to figure out the most practical way to grow a plant, we should probably take their advice on growing the minds of our children with a grain of salt.

If I'm reading the linked page correctly, these metrics were verified by correlating them with individual student improvement on a variety of standardized tests and "more cognitively challenging assignments in math and english". This is better than pure circular logic, but until <i>those</i> metrics are validated, these are suspect. In particular, everything seems to test for shallow understanding.

The title of this article, and perhaps the researchers' conclusions from their data, is wrong. They didn't find the criteria of a great teacher (that is, what makes a good teacher). They just found some correlations among the _qualities_ of good teachers. It's the total opposite meaning. They found results of good teachers, not causes.

I believe that studies like this are well-intentioned in that those conducting the research are truly trying to improve the state of education in the US. However, the result of their work I'm afraid is the 'quantification' of teaching as a profession.

Using test scores in order to judge the quality of a teacher is like using brush strokes in order to measure the quality of a painter.

kudos to the WaPo for giving a link to the study. too often, many publications discuss a study but never link to it, which then makes it harder to go and review it for yourself.

"Test scores"<p>Brilliant. Well, yes, that is the best way to measure a teacher's efficacy, if your goal is to teach students to jump through hoops and pass standardised tests rather than to think.<p>The best teacher I ever had was my grandmother - she taught me the most important lessons I'd ever learn, such as:<p>"Observe the world around you."<p>"Reach your own conclusions."<p>"What was fact yesterday may be fiction tomorrow."<p>Tests are a distraction from learning.