Professor de Freitas from time to time advises the NZ Climate Science Coalition, but he does not speak for it. Nevertheless, this op-ed in today’s Herald gives such a clear view of the issues touching our court case that it deserves a hearing here.
One assumes scientific analysis is objective, so it may come as a surprise that this was challenged in a New Zealand High Court case, the results of which were released last week.
The New Zealand Climate Science Education Trust (NZCSET) contested the claim by the National Institute of Water and Atmospheric Research (Niwa) that New Zealand air temperatures had climbed by 0.9°C over the past century. The trust maintains that objective analysis of the data shows a trend closer to 0.3°C per 100 years.
Recent temperature trends were not in dispute. The court case centred on the fact that the temperature record before about 1965 had been adjusted downwards, a fact not denied by either party. The dispute was over the size of the downward adjustment.
There is little doubt the average annual temperature in New Zealand has been generally trending upwards in line with the expectations of many climate scientists. The question is whether all or part of the warming can be linked to artificial influences.
Long air temperature time series are more often than not beset with artificial, usually human-caused, discontinuities that contaminate what would otherwise be a near-natural record; for example, those sudden discontinuities caused by the relocation of weather stations, changes in instrumentation and observing practices, or gradual changes around the weather station that affect thermal conditions such as the growth or removal of vegetation and, in particular, urban growth and development leading to what is known as the urban heat island (UHI) effect.
Whether sudden or gradual, these changes can introduce inhomogeneities into the long-term temperature time series that distort or even hide the true climatic signal. Uneven spatial sampling because of irregular geographical distribution of weather stations also contaminates the temperature record.
The best-documented example of data contamination is the UHI effect in which data from urban stations are influenced by localised warming because of asphalt and concrete replacing grass and trees. There is a proven close correlation of city population size with urban heating influence on air temperature, which can account for an urban area being as much as 12°C warmer than its rural surroundings. Many studies by climatologists have demonstrated that very small changes in population are enough to induce a statistically significant local warming.
The science of climate change depends entirely on reliable data, quality controlled and homogenised rigorously. Adjusting the data to achieve the reliability required is difficult and controversial. There are other problems.
Temperature trends detected are small, usually just a few tenths of one degree Celsius over 100 years, a rate that is exceeded by the data’s standard error. Statistically this means the trend is indistinguishable from zero. Moreover, trends and temperature differences justified to one or two decimal places and significant figures are unreliable since the amounts are greater than the accuracy of the data allows, and multiple averaging of measurements does not make it more reliable.
Climate services of various countries provide clients with statistical information on climatic variables that is based on long-term observations at a collection of different weather stations. The importance of this statistical material stems from their widespread use as a major input for a large number of societal design and planning purposes, including setting greenhouse gas emissions policy and the economic consequences that follow. For these reasons it is important that climate services deliver the best estimates possible.
The NZCSET’s lawyer summed it up when he told the court the trust was not asserting climate warming did not exist, “we’re saying let’s at least make sure that evidence of this for New Zealanders is accurate”.
Despite the research work undertaken so far, there have been few attempts globally to reassess quantitatively the nature and reliability of homogeneity adjustments to complete national data sets. The High Court case highlights the situation in New Zealand where there have been no peer-reviewed science-based efforts to do this. A court ruling is no substitute.
Argument from authority has no place in science. This was the basis of NZCSET’s case. Argument on the scientific facts and methods used in analyses must now take place. The question is: will it?
Views: 475
Following this with great interest from Canada, keep up good work, well written op-ed.
De Freitas’ piece in the Herald has certainly brought the nutters swarming out of the woodwork! I thought he made his point succinctly and clearly, but it seems that the scientific method is not to be tolerated by the doomsayers.
Yes, AK, plenty of nutters commenting on his article, with one even suggesting he “put a gun to his head”, which caused me to Report.
I agree though, his article seemed reasonable and fair, with this paragraph neatly summing up:
Despite the research work undertaken so far, there have been few attempts globally to reassess quantitatively the nature and reliability of homogeneity adjustments to complete national data sets. The High Court case highlights the situation in New Zealand where there have been no peer-reviewed science-based efforts to do this. A court ruling is no substitute.
There are quite a few commenters on the Herald thread claiming that the BEST data vindicates NIWA’s work, which as Bob pointed out yesterday is untrue.
It would be interesting to get a bit more insight into the BEST methodology and the data used.
There must still be issues with the paucity of data for the early part of the 20th Century that no amount of “kridging and stratification” can deal with.
I actually agree with you Andy, believe it or not. Whenever I have looked at mapped kriging data about some attribute that I think I know something about, I see interpolation which I think is spurious. The early data (say pre-1920?) will probably be utilising data from weather stations in other countries to map the surface. I have no idea how reliable this would be for a small geographically isolated country with few measurement points. You would think that the true value would be somewhere within the confidence interval though, assuming that there are no undetected biases.
Simon, I don’t profess to know anything about Kridging but I do work with data and code for my job, so I tend to be sceptical about “black box” algorithms
It would be interesting to get a better understanding of how BEST comes to its conclusions with respect to NZ.
If the Coalition were to put out, in a peer-reviewed paper, the results of the analysis using the methodology used worldwide, and the Minister could be induced to release the analyses and reports he earlier stated, would this not suffice to bring the official NIWA results into question?
Surely there is a legitimate alternate view of the NZ temperature record to be published here. Surely some MP could be induced to get a response to the Minister regarding his earlier promise.
Doug that’s a brilliant comment
Totally support this bit ‘If the Coalition were to put out, in a peer-reviewed paper, the results of the analysis using the methodology used worldwide…’
Absolutely agree, in fact if the “skeptic” machine would (could) only put out lots of peer reviewed stuff in reputable publications they could seriously challenge the present scientific consensus, it’s only that damn scientific process that gets in the way.
What “machine” is this?
Why Andrew, do you think it is that Dr Vera Power, Manager, Science and Evaluation, NZ Ministry for the Environment, Climate Change Office, ignores published papers that refute papers she cites e.g. Santer08? Not Team approved?
And why does she ignore papers that don’t fit the MfE CC narrative e.g from my communications with the CC Office.:-
***********************************************************************************************************
Citations
Dr Power states in the MftE CC letter to me dated 27 July 2011:-
“In formulating policy on climate change we use peer reviewed science as our evidence base. Much of what you present here comes from the climate change science web site which is neither invigilated or peer reviewed. Many of the ‘issues’ raised there have been dealt with in peer reviewed literature.”
No, the ‘issues’ are being dealt with in an on-going paper – rebuttal – reply – new revised paper – rebuttal – reply sequence i.e. the normal process of science at work. I have already shown previously that papers cited by Dr Power that supposedly “dealt with” certain ‘issues’ have in turn been refuted (see previous correspondence and list below).
I too have used “peer reviewed science as [my] evidence base” in support of my case. The use of previously authored articles is merely to save time (it’s all been written before somewhere) and to cobble the narrative together in a coherent and consistent fashion.
So let’s look at the peer-reviewed literature that I cite either directly or indirectly that Dr Power discounts in favour of a perception that “[m]uch of what you present here comes from the climate change science web site which is neither invigilated or peer reviewed”:-
Vonder Haar et al, 2005
Solomon et al, 2010
Paltridge et al, 2009
Pierce et al, 2006
Minschwaner and Dessler, 2004
Miskolczi, 2010
Scafetta, 2010
McKitrick, McIntyre and Herman, 2010 (Refutes Santer et al 2008)
Douglass et al, 2007
Karl et al, 2006 (radiosondes didn’t find the hotspot)
Allen and Sherwood, 2008 (wind shear proxy for temperature?)
Sherwood et al, 2008 (refuted by Christy et al 2010)
Thorne, 2010 (merely an incomplete review)
Dessler, 2010 (failing under intense scrutiny, actually supports the SB11 rebuttal of it)
Gero and Turner, 2011
Hale and Querry, 1973
Woodworth, 1990
Douglas, 1992
Gregory et al., 2001
Ekman, 1988
Woodworth et al., 1999
Church and White, 2006
Hannah, Bell and Paulik 2011
Watson, 2010
Cole, 2010
Geherels et al., 2008
Bindoff et al, 2007
Knox-Douglass 2010 (refutes Lyman, 2010, Trenberth-Fasullo, 2010)
It is immaterial that a website article “is neither invigilated or peer reviewed” if the text references peer-reviewed literature (32 papers in my case) in support of the argument and I use several web articles including the IPCC AR4 WG1 report that is not independently reviewed (certainly invigilated) and merely does exactly what I have done. Dr Power’s criticism therefore, is without standing. I also challenge Dr Power to state which of the above list she in her MftE CC capacity as Manager: Science and Evaluation or the CC Office has actually evaluated. My question in regard to this is:-
Question 1): Does the MftE CC Office wish to continue to characterize 32 peer-reviewed papers in this way: “[m]uch of what you present here comes from the climate change science web site”.
************************************************************************************************************
At this point the exchange with the Office mysteriously broke down.
That’s just to cite a few and there’s been a stack since then, one of the more damning wrt SLR being Boretti 2012.
Pingback: NZ Herald’s turn to offer propaganda as opinion – De Freitas’ links to cranks hidden from readers
BEST doesn’t provide data for New Zealand. Nor do GISS, HadCRUT or NOAA. They all do grid-boxes covering the entire world’s surface, and New Zealand is not square-shaped.
The figures being quoted either omit a large chunk of the country or include masses of sea water.
In any case, none of the international series which try to calculate global averages have their own datasets. For this part of the world they get their data from NIWA. And Jim Salinger sent them his adjustments years ago.
The average temperature of Dunedin Leith Valley 1887 -1908 from NIWA’s CliFlo database is 12.5 C but NZ BEST temperatures for that era are in the order of 10.5 -11 C:-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Trend.png
BEST NZ is a huge 1.5 – 2 C cooler than CliFlo Dunedin and raises important questions about the validity of the early period of the BEST NZ series.
In addition, the sole station used by BEST for the period 1841 – 1853 is OUTSIDE the New Zealand main land mass (3 islands) but somewhere within the Region of Oceania. Given that BEST shows that era to be in the order of 10.5 C I call bogus. To get that level of temperature the sole station used would have to have been well south of Dunedin, possibly a whaling station somewhere like the Auckland Islands south of New Zealand.
I’ve emailed the Berkely Earth team (not Richard Muller specifically) at their email address on their Contact page here:-
http://berkeleyearth.org/contact/
My communication:-
************************************************************************************************************
Hi Team,
There’s some discussion in New Zealand after the NZCSET v NIWA judgment of the BEST New Zealand series presented here:-
http://berkeleyearth.lbl.gov/regions/new-zealand
The BEST NZ series is said by some to corroborate NIWA’s NZ 7 and 11 station series but I have 2 questions:
1) What is the single station used by BEST for the New Zealand series period 1841 – 1853?
2) What are the two stations used by BEST for the New Zealand series period 1853 – 1863?
History of BEST temperature stations here:-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Counts.png
The station in 1) is not a NZ mainland station but is sited somewhere in the Oceania Region, I’m curious where given the average temperatures from that station are in the order of 10.5 – 11 C but the average temperature from Dunedin Leith Valley (lower half of South Island) from NIWA’s CliFlo database is 12.5 C i.e. 1.5 – 2 C warmer for the period 1887 -1908.
For comparison, the latest NiWA 7SS annual means are 13.1 2010 and 12.8 2011. There’s little difference therefore between CliFlo Dunedin late 1800s era average temperature (12.5) and New Zealand-wide NIWA 7SS 2000s era average temperature (12 .8) but BEST exhibits considerable discrepancy with the early Dunedin data (11).
BEST NZ series: here-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Trend.png
The exact locations of the stations in 1) and 2) would go some way to resolving this discrepancy so I would appreciate your help in doing this.
Regards
Richard Cumming (NZ)
I’m making sure that this issue is in comments under the de Freitas article:-
http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10833106##
My latest comment posted last night (soon after a couple that have been cleared) is still in moderation. That is the one with the CliFlo Dunedin Leith Valley temperature comparison with BEST.
Hopefully the BEST team response will be soon so my posting of it at NZ Herald will get some eyes at least.
I’ve subsequently asked:-
Could you also provide the ‘Region plus 2000 km’ stations (up to about 8 of them) used for the 1853 -1863 period please?
Thanks Richard.
By the way, your comments are all appearing on the Herald article.
I reported the one that describes de Freitas as “an academically fraudulent clown” and made a reference to the Climategate 2 emails.
Maybe the message will get though if we keep reporting abusive comments
Comment count is 45 right now, I still have one I made last night to be cleared Andy.
Nothing forthcoming from the BEST team to date which is a little disconcerting. I suppose good things take time.
Either that or they find themselves in a difficult situation………
Herald comment count is 43 not 45. I still have the Dunedin v BEST comment in moderation but it looks like comments are closed.
The wheels are turning. Received this from Steve Mosher:-
*******************************************************************
Hi,
Elizabeth Muller has informed me that you have some questions about
The Berkeley Earth Data.
How can I help you
Steve
*******************************************************************
To which I repeated (copied) exactly what I had asked via the BEST Contact address (that Elizabeth Muller has obviously seen).
Steve Mosher’s first reply
***********************************************************************************************************
Richard,
Please find attached a couple of files I made for you.
1. is an inventory of the 3 stations in NZ for this early period. You will
note that stations consist of several different reports.
2. the data for these 3 stations.
A couple of notes: the data is the “raw” data that is fed into the estimation
code, so there may be additional QA steps and weighting steps that get
applied. You cannot simply average them.
The chart on the web is the output after these procedures are applied.
Please find below the code for building these files. I have supplied an
R package on the web so that users can answer these types of questions
for themselves. You should be able to find the stations within 2000km
of NZ before 1850 fairly easily, one of them I know would be hobart.
library(BerkeleyEarth)
Directory <- choose.dir()
Data <- readBerkeleyData(Directory=Directory)
Stations <- readSiteComplete(Directory=Directory)
dex -50 & Stations$Lat 160 & Stations$Lon <180)
NewZeeland <- Stations[dex,]
Zid <- unique(NewZeeland$Id)
Data <- Data[Data[,"Date"] < 1864,]
DataIds <- unique(Data[,"Id"])
common <- intersect(DataIds,Zid)
EarlyNZ <- NewZeeland[NewZeeland$Id %in% common ,]
EarlyNZData <- Data[Data[,"Id"] %in% common,]
write.csv(EarlyNZ, "EarlyNz.csv")
write.csv(EarlyNZData, "EarlyNzData.csv")
2 attachments —
EarlyNz.csv http://dl.dropbox.com/u/52688456/EarlyNz.csv
EarlyNzData.csv http://dl.dropbox.com/u/52688456/EarlyNzData.csv
************************************************************************************************************
My response:-
************************************************************************************************************
Thank you Steve.
This gives me something to chew on but I still don't have the sole NZ
station used for the period 1841 – 1853. Could you provide this please
because it will take a while for me to get to grips with the R package
(never used R before although I can enlist the help of others if I
can't drive the package I think).
The BEST New Zealand page I linked to shows that there were 2 stations
used for the period 1853 – 1963 but your EarlyNZData file indicates
that only Musselburgh was used. Is there an error in this graph then?
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Counts.png
Also I note the BEST average of Dunedin Musselburgh is 10.5 C but the
NIWA CliFlo average for Dunedin Leith Valley is 12.5 C. This seems
highly irregular but I can extract the raw Musselburgh data from
CliFlo to see how much it has been adjusted for BEST (if it has). I
think I recall that Leith Valley was closed because it may have been a
little warm but not to that degree I don't think.
Once again, thanks for your assistance Steve.
Richard
Cc’s for this exchange:-
Elizabeth Muller , Robert Rohde
Bob, have you got your eyes on this thread?
BEST apparently uses Dunedin Musselburgh for 1853 -1863 (from EarlyNZData and EarlyNZ sent to me by Mosh) but from Global Historical Climatology Network – Monthly v3 and not from CliFlo.
The CliFlo search for 1800s era returns these Dunedin stations:-
Start_Date End_Date Percent_Complete Name
01-Oct-1862 30-Apr-1886 100 Dunedin,Roslyn
01-Nov-1852 31-Jul-1864 100 Dunedin, Princes Street
01-May-1886 31-Jan-13 100 Dunedin, Leith Valley
No Musselburgh. A CliFlo search for Musselburgh returns:-
15752 I50954 01-Sep-1997 31-Aug-2012 100 Dunedin, Musselburgh Ews
5402 I50951 01-Feb-1947 31-Aug-1997 100 Dunedin,Musselburgh
Global Historical Climatology Network – Monthly v3 here:-
http://www.ncdc.noaa.gov/ghcnm/v3.php
Looks like a mission, better accessed using Mosh’s R code package I’m pickin. That’s here:-
http://cran.r-project.org/web/packages/BerkeleyEarth/index.html
Sigh, looks like I’ll have to learn to drive this. What are your thoughts Bob?
BTW some relevant stuff on Mosh’s blog:-
http://stevemosher.wordpress.com/2012/02/21/berkeley-earth-surface-temperature-v1-5/
Hi Richard,
Yes, it is a bit of a mission, but I’ve got that data somewhere. I’ll have to look for it, so it may take a day or two (the external hard drive it’s on is somewhere else at the moment).
Also Bob, I can’t get any rows for Dunedin Princes Street, I want to see what the average temperature is for that station to compare with Leith Valley and BEST. I’ve entered the following:-
Monthly/Annual Statistics (codes): 02
Station agent number(s): 22645
Start date (yyyy): 1852
End date (yyyy): 1864
No Rows. I’ve checked Data Availability for Agent: 22645, says this:-
I50850 M 02 00 1852-11 1864-07 141 100 Mthly: Mean Temp
http://cliflo.niwa.co.nz/pls/niwp/wstn.data_availibility?cagent=22645
Also Station Details for Agent: 22645 (not much to go on):-
http://cliflo.niwa.co.nz/pls/niwp/wstn.stn_details?cAgent=22645
Any clues Bob?
“I’ve got that data somewhere. I’ll have to look for it, so it may take a day or two”
Cheers Bob, I can wait. Meantime I’ll have a go at the R package.
“Also Bob, I can’t get any rows for Dunedin Princes Street”
Duh! I was logged in as “Public”, forgot I had a subscription, username and password to login with.
Dunedin CliFlo averages:-
Leith Valley 1887 -1908 (21 years): 12.5 C
Roslyn 1866 – 1883 (18 years): 10.5 C
Princes Street 1853 – 1863 (11 years): 10.0 C
Musselburgh 1948 – 1996 (49 years): 11 C
I seemed to recall that Leith Valley was a warm station but I didn’t think it was THAT warm.
BUT I MADE AN ERROR IN MY CALCULATION OF MUSSELBURGH GHCN 1853 – 1863 AVERAGE IN MY REPLY TO MOSH UP-THREAD. THE ACTUAL AVERAGE IS:-
Musselburgh GHCN 1853 – 1863 (11 years): 12.7 C
That’s the same as Leith Valley and MUCH warmer than the BEST NZ graph at that time showing around 11 C.
Whew!
Latest exchange with Steve Mosher worth reporting
*************************************************************************************************************
Richard,
The count at 1841 is 1. The color of the line at 1841 is purple. The purple line is for stations within 2000km.
That station is probably Hobart in Australia, not a station in NZ
The chart represents those stations that make it into the average.
You might want to look at the multi value data.
Steve
************************************************************************************************************
My reply
************************************************************************************************************
Steve,
“The count at 1841 is 1. The color of the line at 1841 is purple”
Yes I’ve realized that from the start but I’ll have to chase that station down myself I guess. If Hobart say, can be the sole NZ proxy for 1841 – 1853 it may as well be the proxy for the rest of the series.
“The chart represents those stations that make it into the average”
This is what I can’t make out wrt 1853 – 1863. The chart, ignoring ‘Region plus 2000 km’, shows station count 2 (‘Within Region’, blue line):-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Counts.png
But the files you have provided (EarlyNZData and EarlyNZ) only reveal 1 NZ mainland ‘Within Region’ station, Dunedin Musselburgh GHCN. Does this mean there is another non-New Zealand mainland (or even mainland) ‘Within Region’ station used for 1853 -1863 that your EarlyNZData database search has not extracted?
Or is there an error in the chart?
Richard
************************************************************************************************************
Steve Mosher is a Database Analyst with BEST:-
http://berkeleyearth.org/steven-mosher/
Mosh’s tone getting obstreperous when challenged, latest exchange (abridged):-
*************************************************************************************************************
There are two stations: auckland and dunedid.
Both are in the inventory stated as starting in 1853. both are in the data
On Sat, Sep 15, 2012 at 7:15 PM, Richard Cumming wrote:
Steve,
“The count at 1841 is 1. The color of the line at 1841 is purple”
Yes I’ve realized that from the start but I’ll have to chase that
station down myself I guess. If Hobart say, can be the sole NZ proxy
for 1841 – 1853, it may as well be the proxy for the rest of the
series.
Well, that is unclear. Basically because of the long correlation length scale information
from all surrounding sites is used to build an an estimate. This is not averaging so you
really have to put all those notions out of your head. The entire field is estimated simultaneously
such that the weather noise is minimized. To create the charts a shape file is used for
NZ and the value of the field is extracted. Another way to look at this is as follows.
During the 1950-present a correlation structure is created. such that If I know the temperture at
location X, I can predict the temperature at location Y within certain bounds. The assumption
in the error bounds are that this structure does not change over time.
“The chart represents those stations that make it into the average”
This is what I can’t make out wrt 1853 – 1863. The chart, ignoring
‘Region plus 2000 km’, shows station count 2 (‘Within Region’, blue
line):-
18603: 1853
18647: 1853
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Counts.png
But the files you have provided (EarlyNZData and EarlyNZ) only reveal
1 NZ mainland ‘Within Region’ station, Dunedin Musselburgh GHCN. Does
this mean there is another non-New Zealand mainland (or mainland)
‘Within Region’ station used for 1853 -1863 that your EarlyNZData
database search has not extracted?
No the files I provided you show 2 new zeeland stations beginning in 1853
[Mosh then copies almost the entire EarlyNZ file which is irrelevant unless the stations actually contribute to the BEST NZ data in EarlyNZData]
************************************************************************************************************
My reply:-
************************************************************************************************************
Steve you say:-
“There are two stations: auckland and dunedid. Both are in the
inventory stated as starting in 1853. both are in the data”
No they are NOT both in the data, only Dunedin 18603 is. Here’s the
1853 – 1863 data from EarlyNZData:-
[Many lines of data deleted. Available from R. Cumming on request. – RT]
You say:-
“This is not averaging so you really have to put all those notions
out of your head”
There’s a number of people I’m sharing this with that will probably
disagree Steve.
You say:-
“No the files I provided you show 2 new zeeland stations beginning in 1853”
No, ONE FILE you provided (EarlyNZ) shows 2 NZ stations beginning in
1953 but the other OTHER FILE (EarlyNZData) has only 1 NZ station
18603 providing data to the BEST NZ series as I’ve proven by copying
the data above.
I repeat, is there an error in the chart or should EarlyNZData have
both Dunedin AND Auckland data which it does not have currently?
Richard
If Hobart is the sole BEST NZ proxy station for 1841 – 1853 as Mosh surmises (not confirmed yet) then let’s have a look at Hobart data from ACORN – SAT here:-
http://134.178.63.141/climate/change/acorn-sat/#tabs=1
Starting with Max data here (Min later):-
http://134.178.63.141/climate/change/acorn/sat/data/acorn.sat.maxT.094029.daily.txt
Max average 93 yrs 19180101 – 20111231: 20.365
Max average 30 yrs 19180101 – 19480101: 17.254
Max average 33 yrs 19480101 – 19811231: 25.414
Max average 30 yrs 19811231 – 20111231: 17.754
Houston, we have a problem.
I can see why Steve Mosher is starting to get frustrated. Why don’t you download the data and find out yourself rather than wasting Steve’s time? The instructions are pretty straight-forward. The datasets are large though.
Simon. I’m only asking for ONE station ID but Mosh is unable at this time to provide it because “My machine is busy on a 2 day data crunch that I cannot interrupt”. The other ID’s asked for have been resolved and provided (the EarlyNZData time sequence jumps around so it’s not that clear).
It would take him “all of 15 minutes” to extract that one ID being a R guy but I’m not so it will take longer (days or more). It doesn’t make much sense for me to get up to R speed for the sake of ONE ID but I will give it a go. Meantime I’ve asked Mosh for the ID when his machine is free and in his time (there’s no rush).
There’s a whole lot more to add since my last update so I’ll have to do another one maybe tomorrow. You have to understand that Mosh is an insider so he’ll be protective when issues arise to the point of diversion (as he’s tried) but the initial request is simple enough and it’s just a matter of cutting through some misunderstanding by both parties – albeit fraught.
but it won’t stop, will it? There will be another question, and then another.
I’m not sure what your issue is. The cross-correlations between stations have been defined by later periods with more data. What you lose in earlier periods is the granualities between regions and obviously some precision. The BEST system will disregard any extreme outliers which may be the source of any data loss.
If you are serious about data analysis, learn R. It is well worth the investment in time. If there is anything you ever want to do, someone will have written an R package to do it. Plus it was originally developed right here in New Zealand, by Ross Ihaka. There are some highly respected associate professors at the University of Auckland.
“I’m not sure what your issue is” – Exactly. The issues are:-
# The appropriateness of Hobart say, as the sole BEST input proxy (prior to kriging) for NZ over the period 1841 – 1853 (see below).
# The degree of adjustment (and the direction) that is applied by the BEST kriging process (that includes non-Region stations) to the input data of mainland Dunedin, Auckland (as it turns out) and another station (that I’ve yet to identify) for the period 1853 – 1863. It’s looking like there’s been a massive downward adjustment (more than 1.5 C) to the Dunedin/Auckland composite, why should this happen? Dunedin and Auckland composite data do NOT need massive downward adjustment and if this is actually the case, the BEST kriging process has resulted in junk output.
I’m working on a Dunedin/Auckland/Other composite series of BEST input data for the period 1853 – 1863 to compare to the BEST output result graph to get an idea of the degree of adjustment (posted later today maybe).
“The cross-correlations between stations have been defined by later periods with more data”
But is there a cross-correlation between ACORN Hobart say (which seems to have been used as the sole NZ proxy 1841 – 1853) and NIWA’s composite NZ 7SS over the period 1918 – 2010?
I don’t think there is and I’m working on a rudimentary comparison (should be posted later today) so that if the ACORN Hobart-NIWA 7SS 1918 – 2010 correlation doesn’t stack up could be extended by others to a statistical correlation test that either validates or invalidates Hobart as a NZ proxy (or whatever actual station was used).
I don’t see the need to become an R expert just to extract ONE ID but maybe I’ll have to if the ID is not forthcoming from the BEST team. I think Mosh will provide it and I’ll just have to wait – no problem.
Hobart continued. I made an error in the 1948 – 1981 Max average so disregard the previous Max figures. Min is omitted for brevity so we go straight to a Hobart Max/Min average comparison to NIWA 7SS for the period 1918 – 2010:-
Average 93 yrs 1918 – 2010: Hob 13.07, 7SS 12.28, Diff -0.79
Average 30 yrs 1918 – 1947: Hob 12.99, 7SS 11.87, Diff -1.12
Average 33 yrs 1948 – 1980: Hob 12.96, 7SS 12.34, Diff -0.62
Average 30 yrs 1981 – 2010: Hob 13.27, 7SS 12.61, Diff -0.66
I’m sure a statistical correlation test would invalidate a Hobart/NZ 1918 – 2010 correlation.
Conclusion alternatives (assuming Hobart is the station used by BEST), either:-
A) The BEST Hobart input proxy for NZ 1841 – 1853 is junk.
B) The NIWA adjustments for the period 1918 – 1947 are junk.
C) A combination of A) and B).
Analysis of the BEST 1853 – 1863 input stations will follow down-thread under a new and separate thread header.
Email exchange updates with Steve Mosher will also follow down-thread under new and separate thread headers so they can be addressed separately.
Interesting thought:-
If NZCSET 7SS values are used for 1918 – 1947 instead of NIWA’s there may actually be a Hobart/NZ correlation i.e. a Diff closer to -0.62 and -0.66 instead of -1.12.
Problem being, there seems to be negligible warming trend in Hobart data 1918 – 2010 compared to when the warming occurs in either the NIWA or NZCSET 7SS (happens prior to 1981) given the representative absolute values for the respective Hobart periods:-
1918 – 1947: 12.99
1948 – 1980: 12.96
1981 – 2010: 13.27
Bob?
Certain people are getting hot under the collar that CdF’s links to us cranks is not stated on the Herald article, as are his other links to “inactivist” (sic) groups
Tattoo this: “It’s the Sun, stupid!”
Written by Willie Soon and William M. Briggs | September 10 2012
New Berkeley BEST project temperature records confirm: changes in solar radiation influence climate
Scientists have been studying solar influences on the climate for over 5000 years.
Chinese imperial astronomers kept detailed sunspot records. They noticed that more sunspots meant warmer weather. In 1801, the celebrated astronomer William Herschel (discoverer of the planet Uranus) observed that, when there were fewer spots, the price of wheat soared. He surmised that less “light and heat” from the sun resulted in reduced harvests.
Earlier last month Professor Richard Muller of the University of California, Berkeley’s BEST project announced that, based on their newly constructed global land temperature record, “no component that matches solar activity” was related to temperature. Instead, Professor Muller said, carbon dioxide controlled temperature.
Could it really be true that solar radiation, which supplies Earth with the energy that drives our weather and climate – and which, when it varied, caused the climate to shift over the ages – is no longer the principal influence on climate change?
Consider the charts that accompany this article. They show some rather surprising relationships between solar radiation and daytime high temperatures, taken directly from Berkeley’s BEST project. The remarkable thing about the graphs is that these tight relationships hold for areas as large as the USA, to areas as small as the Sunshine state, and even as minor as our nation’s capital.
>>>>>>>>>
http://climatechangedispatch.com/home/10457-tattoo-this-its-the-sun-stupid
Chris de Freitas has no claim to objectivity in these matters; in fact, he is an embarrassment to the NZ science community and the University of Auckland.
Whilst editor of a once-respected science journal, de Freitas accepted a paper which claimed that there was nothing unusual about 20th century warming.
Thirteen scientists wrote a rebuttal, one saying the paper was “so fundamentally misconceived and with so many egregious errors that it would take weeks to list and explain them all.”
Five journal editors resigned in protest, and documents later obtained under the US Freedom of Information Act showed that the primary author of the paper had received over $1,000,000 from fossil fuel interests such as Exxon Mobil.
http://en.wikipedia.org/wiki/Soon_and_Baliunas
“…the primary author of the paper had received over $1,000,000 from fossil fuel interests”
Whoop-de-do, Take a look at what Lewandowsky got his mitts on:-
Lewandowsky gets $1.7m of taxpayer funds to denigrate people who disagree with him
The Bottom line:
This kind of unscientific poor standard work would not get attention or have any credibility if it were not funded by the Australian Government. According to his 28 page CV he claims to have been a part of $4.4m in grants.
Nice work if you can get it.
If we do not demand higher standards and turn off the tap filling this well of personal bias dressed as research, we’re letting good scientists down, we’re letting hard working tax-payers down, and we’re letting our children down.
See below for details of the funding…
>>>>>>
http://joannenova.com.au/2012/09/lewandowsky-gets-1-7m-of-taxpayer-funds-to-demonize-people-who-disagree-with-him/
Question is: Who’s paying John Cook?
Nice work if you can get it….
http://www.desmogblog.com/joanne-nova
“DeSmogBlog found that the sponsors of the 2009 conference had collectively received over $47 million from major oil companies and right-wing foundations”
Duh! That $47m wasn’t specifically for their climate section, the actual amount going there was a miniscule fraction of that.
Follow the real climate money (that’s billions Rob):-
http://joannenova.com.au/tag/climate-money/
Indeed it is Richard: the annual revenue of the top 8 fossil fuel companies is in excess of US$ 2,560 billion
http://en.wikipedia.org/wiki/List_of_companies_by_revenue
Globally, fossil fuel subsidies are about US$ 400 billion , which is six times that of renewable energy subsidies.
http://www.bloomberg.com/news/2011-11-09/fossil-fuels-got-more-aid-than-clean-energy-iea.html
That is serious money that will buy you a lot of conservative think tanks, commentators and “skeptic” websites….
http://www.greenpeace.org/usa/en/campaigns/global-warming-and-energy/polluterwatch/koch-industries/
“…..fossil fuel subsidies are about US$ 400 billion” – Bzzzzt Wrong
Wild claims of “fossil fuel subsidies” debunked
http://joannenova.com.au/2012/09/government-burn-70-billion-a-year-subsidizing-renewables-and-wild-claims-of-fossil-fuel-subsidies-debunked/
Just occurred to me that ‘Statistical Audit of the NIWA 7-Station Review’ would most appropriately be published in a statistical journal.
There’s a list of them here:-
http://www.statsci.org/jourlist.html
Includes:-
Australian and New Zealand Journal of Statistics, Blackwell Scientific (Oxford), for the Statistical Society of Australia Inc.
International Journal of Climatology, Wiley (New York)
Journal of Agricultural, Biological, and Environmental Statistics, American Statistical Association and International Biometric Society. First published 1996.
Journal of Applied Meteorology, American Meteorological Society (Boston). Full text online.
Australian & New Zealand Journal of Statistics
Vol 54 (4 Issues in 2012)
Edited by: Stephen Haslett, Michael Martin, Martin Hazelton, Alan Welsh
Print ISSN: 1369-1473 Online ISSN: 1467-842X
Published on behalf of New Zealand Statistical Association, Statistical Society of Australia, Statistical Society of Australia
Impact Factor: 0.436
Read Now Online
Subscription Information
* Description
The Australian & New Zealand Journal of Statistics is divided into two sections. The Applications Section publishes papers demonstrating applications of statistical techniques to problems faced by users of statistics in the sciences, government and industry. The Theory and Methods Section publishes papers containing original contributions to the theory and methodology of statistics, econometrics and probability.
The Applications section gives preference to papers featuring the use of new statistical procedures or the extension of existing statistical techniques to novel areas of application. It seeks to aid teachers of statistics by placing statistical methods in context.
The Theory and Methods Section seeks papers motivated by a real problem and which demonstrate the proposed theory or methodology in that situation.
http://au.wiley.com/WileyCDA/WileyTitle/productCd-ANZS.html
Two options:-
1) Email the ‘Statistical Audit’ to the Australian & New Zealand Journal of Statistics say, along with the three reviews. Having been already reviewed, it would be unnecessary for the Journal reviewers to review it too. The Audit could be published in the next volume if it was accepted on that basis.
2) As for 1) except that the three professionals are asked if they will become co-authors of the ‘Statistical Audit’ and if they do the Audit would go through the Journal’s normal review process.
NZ mainland BEST inputs BEFORE KRIGING 1853 – 1863:-
Ave Dunedin 12.69
Ave Auckland 15.06
Ave Dun/Auc 13.88
Wellington is used for a year but that’s insignificant.
First we compare that 13.88 figure to the average of the first 10 years of the NIWA 7SS that have all 7 locations 1913 -1922 which is 11.93, Diff (13.88 – 11.93) is -1.95 C.
Therefore, an 1853 – 1863 Dunedin/Auckland composite NZ proxy is 1.95 C warmer than the first full 7 station decade of the NIWA 7SS – NIWA has been found out here.
Now we compare that 13.88 figure with the BEST output AFTER KRIGING that includes the influence of ‘Region plus 2000 km’ stations:-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Trend.png
In the order of 10.5 C, Diff (13.88 – 10.5) is -3.38 C .
The BEST ‘Region plus 2000 km’ stations used as input to their kriging methodology (reasonable in it;s own right – NIWA offers a similar NZ product) produces a huge and unnecessary -3.38 C downward adjustment to the mainland (Dunedin/Auckland) stations.
I call bogus on both counts.
If you can keep Steven Mosher onside then he will be able to help you. after all, he played a fairly prominent role in the UEA FOIA saga.
He is hardly likely to be fudging data or hiding it from people
“If you can keep Steven Mosher onside” – Yes but as things progressed he became less and less “onside” to the point of blowing the simple request out of all proportion, but I think it’s all been resolved. Just that one 1841 – 1853 ID outstanding and I think Steve will provide it when he has time and his machine is freed up from a data crunch. Meantime I’ll assume it’s Hobart (my suspicion anyway) and plug away at the R package maybe (I have a life but R is not a priority in it).
He’s VERY defendant of BEST methodology saying I “misunderstand” kriging but I’m not actually questioning kriging as I tried to explain to him (NIWA has been offering a similar product at a price for some time now), I’m questioning the validity of the early NZ inputs and the output resulting from them – GIGO.
My misunderstanding was because the R code spits out the stations in a one column sequence to EarlyNZData but I’ve been used to seeing the NIWA 7SS stations formatted one column each in parallel across a spreadsheet which is much clearer. The EarlyNZData time sequence starting at 1853 is Dunedin then Wellington to 1863 but then it jumps back to Auckland at 1853 so that the 2 stations starting then are Dunedin and Auckland. I didn’t notice the Wellington to Auckland date change in the sequence.
I’ll do a synopsis of the exchange to date at some stage but not as I’ve been doing it, the email count is 21 at the moment.
* Site name: HOBART BOTANICAL GARDENS
* Site number: 094030
* Latitude: 42.87 °S Longitude: 147.33 °E
* Elevation: 27 m
* Commenced: 1841 Status: Open
* Latest available data: 31 Jul 2012
http://www.bom.gov.au/climate/averages/tables/cw_094030.shtml
Set ‘Period’ to “1841-1870”, view ‘Annual’. For 1841 – 1854 (14 years):-
Mean maximum temperature (°C): 17.0
Mean minimum temperature (°C): 7.8
Average mean temperature (°C): 12.4 (calc)
Compare to BEST NZ output graph:-
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Trend.png
In the order of 10.5 C, Diff (12.4 – 10.5) is -1.9 C.
Summarizing means for the period 1841 – 1854:-
12.4 BEST NZ input from BOM Hobart Botanical Gardens (assumed)
10.5 BEST NZ output (Diff -1.9)
Compare to:-
13.88 BEST NZ input from GHCN Dunedin/Auckland composite 1853 – 1863
10.50 BEST NZ output 1853 – 1863 (Diff -3.38)
11.93 NIWA 7SS 1913 -1922
I still call bogus.
Hobart Botanical Gardens, Latitude: 42.87 °S
Kaikoura South Island NZ, Latitude: 42.41 °S
“It’s déjà vu all over again” – Yogi Berra
Compare:-
13.88 BEST NZ input from GHCN Dunedin/Auckland composite 1853 – 1863
13.10 NIWA 7SS 2010
12.80 NIWA 7SS 2011
Compare:-
13.88 BEST NZ input from GHCN Dunedin/Auckland composite 1853 – 1863
12.69 NIWA 7SS 2001 – 2011 (Diff -1.19 C)
It gets worse.
Compare:-
11.93 NIWA 7SS 1913 -1922
10.80 BEST NZ 1913 -1922 (approx) (Diff -1.13 )
12.69 NIWA 7SS 2001 – 2011
11.40 BEST NZ 2001 – 2011 (approx) (Diff -1.29)
http://berkeleyearth.lbl.gov/auto/Regional/TAVG/Figures/new-zealand-TAVG-Trend.png
What a joke, the NZCSET 7SS is waaaay closer to NIWA’s 7SS than BEST NZ.
1910 NIWA NZCSET Diff +0.5 (-1.13 NIWA BEST)
2010 NIWA NZCSET Diff -0.01 (-1.29 NIWA BEST)
http://i54.tinypic.com/27xjm0k.png
So BEST NZ, GHCN Dunedin/Auckland composite, BOM ACORN-SAT Hobart, and, NIWA 7SS disagree variously and wildly.
The sheer bogosity of it all has now reached stupendously bizarre proportions.
Let’s see, Richard, whose financial analysis has more researchers and credibility?
I’ll give you a clue: on the one hand we have Bloomberg:
whereas, on the other, we have Joanne Nova, an internet blogger.
I know who you would prefer to believe, but I also know you believe in huge invisible undersea volcanoes….
Quick, look, there’s another unicorn!
http://en.wikipedia.org/wiki/Bloomberg_L.P.
Quick Rob, attack the person not the issue that’s the (warmist) way isn’t it?
When the person is unable to come up with a coherent fact-based argument, what other target is there?
So, do you seriously believe that Jo Nova’s financial data analysis is better than Bloomberg’s??
“When the person is unable to come up with a coherent fact-based argument”
Rob you haven’t even addressed the argument so how do you know it isn’t coherent and fact-based?
Reminds me of a certain Amazon book review that I can’t quite recall exactly (what was it Andy?).
Here it is:-
1.0 out of 5 stars
Lies, misrepresentations, and a bible for climate change deniers,
October 16, 2011
By Peter Gleick “PGleick”
This review is from: The Delinquent Teenager Who Was Mistaken for the World’s Top Climate Expert (Kindle Edition)
This book is a stunning compilation of lies, misrepresentations, and falsehoods about the fundamental science of climate change. It compiles the old arguments, long refuted, about the Intergovernmental Panel on Climate Change, which summarizes the state of science on climate change. The IPCC reports — the most comprehensive summary of climate science in the world — are so influential and important, that they must be challenged by climate change deniers, who have no other science to stand on. LaFramboise recycles these critiques in a form bound to find favor with those who hate science, fear science, or are afraid that if climate change is real and caused by humans then governments will have to act (and they hate government).
Are you already convinced that climate change is false? Then you don’t need this book, since there is nothing new in it for you.
If you respect science, then you ALSO don’t need this book, since there’s no science in it, and lots of pseudo-science and misrepresentations of science. See, especially, the section trying to discredit the “hockey stick” — long a bugaboo of the anti-climate change crowd. Seven independent scientific commissions and studies have separately verified it, but you won’t find out about that in this book.
Really: save your money and battery life.
http://www.amazon.com/review/R3DB7LHRMJ14G5/ref=cm_cr_pr_perm?ie=UTF8&ASIN=B005UEVB8Q&linkCode=&nodeID=&tag=
From Amazon:-
http://www.amazon.com/Delinquent-Teenager-Mistaken-Climate-ebook/dp/B005UEVB8Q
Donna Laframboise’s new book causing reviews in absentia amongst some AGW advocates
http://wattsupwiththat.com/2011/10/16/donna-laframboises-new-book-causing-reviews-in-absentia-amongst-some-agw-advocates/
“The first fun part: Gleick apparently never read the book before posting a negative review, because if he had, he wouldn’t be intellectually slaughtered by some commenters who challenge his claims by pointing out page and paragraph in the book showing exactly how Gleick is the one posting false claims.”
“The other fun part? Gleick apparently doesn’t realize he’s up against a seasoned journalist, he thinks Donna is just another “denier”. Another inconvenient truth for Gleick is that she was a member of the board of directors of the Canadian Civil Liberties Association – serving as a Vice-President from 1998-2001.”
“I also know you believe in huge invisible undersea volcanoes….” – Invisible?
Like this one?:-
Our new island? Pumice float stuns Navy (+video, photos)
http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10826068
And these ones?:-
Submarine Volcanoes
Scientists estimate that at least 80% of the world’s volcanism occurs in the oceans!
http://www.gns.cri.nz/Home/Learning/Science-Topics/Ocean-Floor/Undersea-New-Zealand/Submarine-Volcanoes
Maybe you need your eyes tested Rob.
Yeah, right… the total climate forcing from volcanoes is less than 1% of the anthropogenic CO2 forcing.
http://volcanoes.usgs.gov/hazards/gas/climate.php
If you have research to challenge these figures, then cite it.
So NOW you can see those volcanoes Rob – your blindness remarkably healed, a miracle.
I don’t give a toss about the CO2 emissions Rob. May I remind you that you are on the Defaulters List for failing to refute Eggert’s hypothesis that CO2 has negligible effect above about 200 ppm.
Richard, your word salad reply shows that you cannot cite research to refute the fact that anthropogenic CO2 emissions are less than 1% of total volcanic.
May I remind you that you have a history of “making stuff up”, as per your “Obama quote” above?
IMHO, the rest of your “arguments” are little better than numerology, crossed with glossolalia.
“…you cannot cite research to refute the fact that anthropogenic CO2 emissions are less than 1% of total volcanic”
Why would I? Why should I? When have I ever disputed “the fact” anyway? And it looks like maybe you’ve got it the wrong way around but who cares?
I repeat:-
I don’t give a toss about the CO2 emissions Rob. May I remind you that you are on the Defaulters List for failing to refute Eggert’s hypothesis that CO2 has negligible effect above about 200 ppm.
Court no substitute for science but in Italy the court holds scientists accountable for inaccurate prediction of a real disaster:
‘Scientists Found Guilty In L’Aquila Earthquake Trial’
Seismic experts were sentenced to six years in prison and $10 million each in damages for not accurately predicting Italy’s fatal 2009 L’Aquila earthquake, which killed 308 and injured 1600.
http://www.thedailybeast.com/articles/2012/10/22/scientists-found-guilty-in-l-aquila-earthquake-trial.html
I wonder if climate scientists will be similarly held accountable for an inaccurate prediction of an unreal disaster.
Roger Pielke, Jr. on L’Aquila science via WUWT:-
Another lesson is that debates over forecasts and uncertainty often overshadow knowledge that is far more certain. Paul Somerville and Katharine Haynes of Macquarie University note wryly that “no action has yet been taken against the engineers who designed the buildings that collapsed and caused fatalities, or the government officials who were responsible for enforcing building code compliance.”[6]
The real tragedy of L’Aquila may not be that scientists led the public astray with their bumbled discussion of predictive science but, rather, that our broader obsession with predictions blinds us to the truths right before our eyes.
>>>>>>>
http://wattsupwiththat.com/2012/10/22/pielke-jr-on-lessons-of-the-l%CA%BCaquila-lawsuit-comparisons-to-lessons-learned-on-nws-forecast-falures/#more-72835
Couldn’t be more relevant to the current climate science situation.