In Australia, Warwick Hughes has followed with interest our attempts to obtain from NIWA details of their adjustments to the NZ temperature record. When the Bureau of Meteorology (BoM) supplied a letter apparently certifying the Bureau’s “peer review” of NIWA’s review of the temperature record, he noted our complaint that “there must be more than this.”
Hearing of my request to NIWA for records relating to that review by the Bureau, he was minded to help. So, back in February, he filed a Freedom Of Information (FOI) request with the Australian Information Commissioner.
In response to that FOI request, the Bureau submitted to the Information Commissioner a Schedule of Documents dated 6 May 2011.
Somebody at the Bureau has put in hours of work tracking these documents down, describing them, analysing their relevance to Warwick’s request and assessing whether they met the provisions for exemption. Well done, them.
The schedule describes 159 relevant documents, amounting to several thousand pages, and what do you know? The BoM claims full exemption from the FOI Act in respect of every single page! Who could have predicted that?
The BoM had earlier advised that the Bureau itself had no reason to oppose disclosure of the documents, but it was objecting on an interim basis while it awaited instructions from NIWA. Because NIWA had delayed answering the question, the Bureau requested an extension of time.
The Bureau says it believes its whole transaction with NIWA was subject to a one-way obligation of confidence, because it was “peer-reviewing” NIWA’s research.
But the Schedule doesn’t look anything like the description of a scientific peer review:
- The first 17 documents mainly consist of data and links sent by NIWA, then the next 28 are about statistical tests applied by the Bureau to that data.
- Then NIWA drafted 30 pages of its ‘Review’ (doc 47) which were the subject of numerous discussions and a report by the Bureau.
- About six weeks later, NIWA sent a re-drafted document, now 84 pages (doc 62). The Bureau undertook a series of ‘RH tests’, giving rise to data which were the subject of 24 separate internal reports, covering 40 pages. More than 50 internal documents are listed — all generated before the BoM reported back to NIWA.
- The next step was that NIWA changed all of its station reviews (except Hokitika) and sent 31 pages of tracked change reviews to the Bureau. On 6 December, the Bureau sent NIWA its six-page “review document”. NIWA obtained the Bureau’s written consent to put the final ‘Review’ on its web site.
- More than a week after the ‘Review’ was published, NIWA forwarded seven “final responses to Bureau reviews” (covering 40 pages).
So if it’s not a peer review, what is it?
It looks like a consultant-client relationship.
NIWA starts writing up its review AFTER it obtains the results of a mass of statistical tests carried out by the consultant. Then the consultant really comes into its own, with a flurry of ‘RH tests’ undertaken by its own complex climate model. These inputs cause the client to re-write all the station reviews, sending them back to the consultant for comment – but making its own final decisions on what gets published.
The exemptions claimed by the Bureau against providing the information are justified by the “foreign government” clause.
One intriguing side-play is that the Bureau rejected Hokitika after the RH tests, and deleted reference to that station from its ‘letter of support’. But NIWA included it anyway!
After all this work, it is curious that RH tests receive only one mention in NIWA’s 159-page Review. A footnote to the Masterton section (p 41) notes that: “Further research will determine adjustments to monthly temperatures … and apply statistical methods (RH tests, Wang, et al.) to identify other change-points in the data.”
The biggest thing NEEDED for the ‘Review’ is some justification for NIWA’s maverick practice of making adjustments based on comparisons of isolated stations (see NIWA’s maverick methodology). All the scientific literature to date insists that comparisons be limited to neighbouring stations which are subject to the same local weather conditions – and the Bureau itself (Torok & Nicholls) has previously been strong on this requirement.
In February 2010, the Hon Dr Wayne Mapp, Minister of Science, advised the Parliament that NIWA would publish a justification of its Review methodology in a scientific journal, during “the next fiscal year” (expiring 30 June 2011). Among interested parties, it has been widely expected that NIWA would disclose research to show that comparisons with “highly-correlated non-neighbouring sites” are equally reliable.
If they don’t do that, they leave the NIWA method exposed to severe criticism — a non-scientific orphan.
To achieve publication, NIWA needs somebody to sign off on a genuine peer-review. Who better to choose than NIWA’s closest counterpart – the BoM? (“pal-review” is endemic in the special world of climate science.)
We won’t know for sure until the Australian or New Zealand FOI legislation finally starts working and delivers this public information into the hands of the public who paid for it, but an educated guess is that the extensive BoM statistical/modelling consultancy was intended to support the forthcoming research paper on isolated-station comparisons.
Views: 83
If BOM releases its methods of reviewing and coroborating NIWA data, won’t it be revealing how BOM analysis Australian data? And thereby open itself up to Australian complaints?
It would if it uses the same methods in Australia. But BOM is sensitive to UHI effects, which NIWA is not. And BOM doesn’t use comparisons with non-neighbour stations.
It’s interesting to learn that BOM wrote a 6-page review, and NIWA sent back a 40-page “response” (AFTER it secured BOM’s letter of support). It rather looks like NIWA might have disagreed with the views of its consultant!