OK, here's what I figured out. I was able to reproduce his results pretty well. The two main things I found were that his choice of rural stations tended to double the difference the adjustments made. What he missed was that some of the stations he used were actually 2, 3 or 4 stations merged together. The raw data for these stations are simple merged together in the raw data without any adjustment for the different stations. In my reconstruction, I didn't try to adjust these stations, I just dropped them.
While that corrected some of the "gap" between raw and adjusted rural data, it didn't correct for all of it. There may be other factors that add to the "gap" (like being averaged anomalies instead of grid cells), but I'm not going to bother trying to figure it out. So the next thing I did was look at the data set that was adjusted only for changes in the time of observation. As I mentioned earlier, there was a shift from taking observations midday to taking them early in the morning which would introduce a cooling bias.
When plotted with the rural data it is very clear that the adjustment for the time of observation was by far most of the adjustments made. When plotted with the urban data there was very little difference - not surprising since the raw and adjusted urban data are pretty similar.
So - the question is why does the change in observation time adjustment affect the rural stations more than the urban ones? The only thing I can think of is rural stations tend to be liquid-in-glass thermometers that are manually read while urban stations tend to be electronic and are read automatically. I don't know if the time of observation changed with more rural stations than the urban ones (there is a data file that might indicate that, but I don't understand whats in it).
Here's my plots. I added some offsets to get the charts to line up like the article did (he subtracted 0.2 from the raw rural data to get it to line up more to the left side)
Rural & urban raw data from the article:
My reconstruction of the rural data before and after adjustment using the same stations
Unmerged rural stations before & after all adjustments and adjusted only for time of observation.
Notice that using just unmerged stations reduced the "gap" almost by half. Also notice that almost all of the adjustment was for changes in the time of observation
And unmerged urban stations before & after all adjustments and adjusted only for time of observation.
Notice that the time of observation adjustments were minor.
[edited 07/31/2012 to replace rural station chart that imageshack had dropped]
