<i>So all we need to do in order to follow these stations, is filter out those in the northern hemisphere and re-run the job.</i><p>Isn't this the same sort of thing that the tree ring people did? There were some statistical points that looked like they were going to mess with the results so they threw them out? They only threw out a few outliers, which is standard practice, but here we're throwing out a whole class of data.<p>It's also interesting that the northern hemisphere is warming up faster (if the methodology chosen by the author is correct). Since most of the earth's landmass is on the northern hemisphere (as well as most of the developed countries), doesn't that mean we're in trouble?<p><i>By looking at each station indiviually instead of compressing them to an unevenly weighted average, we will be able to clearly deduce how the weather has changed through the years recorded.</i><p>This still doesn't take into account the density of stations. Also, trying to compare stations like that, with the curves obscuring each other, is a crime of presentation. If you took the time to do a regression on the datapoints with the other data, it surely would not have took much longer to code for regression on each of the individual weather stations (incorrect as that may be).<p>On a different note, the use of clojure was enjoyable, which deserves an upmod.
I'm torn on this article. I'm very appreciative of a real-world example of using clojure to manipulate data but at the same time it treats a trivial and entirely unscientific analysis of one data set as a complete, concrete and comprehensive summary of an entire field of science.
Well, there's a few problems. Gore didn't actually use the Mann hockey stick that is presented, although the hockey stick in 'An Inconvenient Truth' is very simliar (in that it omits the Medieval Warming Period)<p>Those commenting on the start date fail to realise that the start date of the data set he uses is 1929. You can't blame the author for that.<p>The fact that it's part of the 'head' or 'blade' of the stick is irrelevant, all of the 'handle' is based on multi-proxy reconstruction, which has major data problems, as the endless to-and-fro on this issue says. One of the major problems highlighted has been the use of tree-rings as a proxy, and the tree rings do not agree with the instrumental record. This is the famous 'hide the decline', where the authors sought out statistical methods (the trick) in order to splice the tree ring proxies with the instrumental records, which they really shouldn't have done.<p>The field of coming up with global averages from temperature datasets is so complex I won't even try and make a comment on that. Plus I don't know any clojure.
I just tagged this excellent article on delicio.us - description: "Awesome example of using Clojure for data manipulation!"<p>A good read, and I bet that a few of the code snippets in the article will save me coding time.
I have a project based on the Global Historical Climatology Network data, going back 300 years. This is based upon readings from actual thermometers on ground based weather stations, rather than tree rings or ice cores which are far more difficult to decypher.<p><a href="http://code.google.com/p/tempgraph/" rel="nofollow">http://code.google.com/p/tempgraph/</a><p>You can graph the data in various ways, all the way down to individual weather stations.