Today we bring you more details of the High Court hearing from two weeks ago, including a surprising admission by NIWA, who practically discard the “peer review” provided for them by the Australian Bureau of Meteorology (BoM).
In various previous posts (NZ sceptics v. NIWA – summary of case, More about the NZ temperature record, What warming, Incredible sham from NIWA and others) we summarise the *Coalition’s case against the Seven-Station Series (7SS).
Since 1999 this temperature series has been providing the basis for New Zealand’s climate change policies, but its major role has been to be presented whenever a public body needs official evidence of the country’s temperature history.
Until we investigated, NIWA’s web site did not disclose that the temperature readings had been adjusted. We only found out when we went to graph the data they provided – our graph was wildly different from theirs and showed no warming. For years, NIWA scientists even refused polite requests from our scientists to disclose the adjustments they had made to enable independent review of their results.
As our frustration grew, Coalition members and the CCG in 2009 published their landmark review Are we feeling warmer yet, showing the difference between the raw and adjusted readings, and began publicly to criticise NIWA’s reluctance to release official public data.
NIWA promises Parliament they’ll publish
Unable to respond to the Coalition’s criticisms (it transpired later that they had lost the details, they just didn’t tell anyone), NIWA undertook to fully reconstruct the 7SS in a review project which would occupy five or six scientists for several months. In making this promise to Parliament in February 2010, NIWA’s minister (the Hon. Dr Wayne Mapp) added three further undertakings:
- the NIWA work would be externally peer-reviewed by two scientists from the Australian Bureau of Meteorology (BoM).
- the confidence levels of the adjustments would be calculated and disclosed.
- by June 2011, a scientific paper describing NIWA’s methods would be submitted to an academic journal for publication, and would be subject to independent peer reviews.
That made four principal objectives. What’s their status?
- Reconstruction: as we know, it’s finished.
- Peer reviews: NIWA and the BoM say that a single peer review occurred. NIWA claims their report has been approved; the BoM describes a somewhat cursory examination. There’s no evidence of the BoM’s approval and no sign of a thorough examination. There’s no evidence of other peer reviews.
- Confidence levels: unpublished.
- Formal paper: not submitted, unpublished.
In summary: three out of four objectives were not achieved. That is not adequate and I note that no explanation has been offered. It’s certainly not what the Minister promised the Parliament.
After the announcement of the review, nothing happened for several months, so the Coalition issued Judicial Review proceedings.
About six weeks later, NIWA sent station records to the BoM showing their proposed adjustments. After some to-and-fro, NIWA received a final report from the BoM (excluding Hokitika) in early December. NIWA didn’t like it a bit. They first sent off an interim reply and then gave the BoM a 40-page broadside on the eve of Christmas. The BoM didn’t respond.
So NIWA made the best of it. They published a report on the ‘Review of NIWA’s “Seven-Station” Temperature Series’ on 16 December, 2010, with its new temperature record (NZT7). The report included a wishy-washy letter from the BoM confirming it had carried out a peer review, but saying nothing significant about the outcome.
OIA requests refused
In the global climate change debate, much reliance is placed by some, such as the IPCC, on allowing only peer-reviewed material, in an attempt to ensure good quality information. There is also enormous public interest in our suit against NIWA, both here and overseas.
Recognising this, I asked NIWA for copies of their correspondence under the Official Information Act. I’ve previously described the outcome in NIWA’s review taking a hiding, NIWA’s review: what are they hiding? and NIWA — show us the peer review!
The Ombudsmen’s office warned of possible delays in January 2011. They later asked for comment on NIWA’s claim for legal privilege on the grounds that the whole peer-review had been undertaken as a defence to our legal proceedings.
My first request related to the period up to publication of the Review. In July 2011, I lodged a further request for the period subsequent to the Review. I finally received “no” for an answer just a few weeks ago, in July.
Independently, Warwick Hughes asked the BoM for copies of the correspondence under the Australian Freedom of Information Act. His experience was described in What are the Aussies hiding, No, really — what are they hiding, Some questions for the BoM’s FOI executive and other posts.
The Coalition’s legal advisers sought Court-supervised discovery and inspection of the BoM peer-review papers. NIWA’s advisers fought this tooth and nail. After about eight months of constant delays, the parties settled the discovery dispute on a basis which left out the BoM papers.
NIWA and BoM part company
The Coalition then filed an amended claim to include the Review. NIWA’s amended defence repeatedly referred to it, claiming the BoM peer-review “found the results and underlying methodology used for the 7SS were sound.”
Unfortunately, the BoM don’t agree with them. It was time to split.
So at the eleventh hour, buried at paragraph 306 of Dr David Wratt’s revised affidavit, just a few days before the hearing, we found this surprising abandonment of what was previously the highly-valued and much-touted (such as for assuring the Parliament that all was above board) BoM peer review:
306. TJD113 – TJD120 regarding the BoM review of NIWA’s report: I confirm that NIWA and the Australian Bureau of Meteorology (BoM) carried out a peer review process. NIWA does not seek to rely on it for the purposes of this proceeding. The reason for that is not that the BOM peer review did not support NIWA’s approach to adjustments and homogenisation of the raw data in temperature records (it did) but, rather, that NIWA and BOM regard the process of peer review and the interchanges between them as confidential, privileged, and subject to public interest immunity. It is inimical to the future relations between NIWA and international agencies such as BOM for these exchanges to be subject to scrutiny in litigation: it impedes the free and frank exchange of information which underpins the value of the relationship between such agencies. Accordingly, rather than rely on the BOM’s work, NIWA proposes to rely on evidence prepared specifically for the purpose of this litigation. In the event that the plaintiff seeks further discovery (which is suggested in Mr Dunleavy’s affidavit) that application will be resisted on the grounds stated above together with the fact that this matter has already been canvassed and settled during the interlocutory rounds of this case when limited orders for further discovery were made by consent. Those orders excluded the BOM material and, so as to put the matter beyond doubt, NIWA does not rely on the BOM review in any event for the purposes of these proceedings.
This boils down to a confession to the Court that NIWA has no evidence to show that the BoM approves of NIWA’s review. NIWA does not even bother to present the ineffectual BoM covering letter at page 15 of the Review, for it expresses no approval of the report – though NIWA claims it does.
NIWA’s obstructive behaviour
The plaintiff’s solicitor responded to NIWA’s last-minute renunciation with these comments:
NIWA now goes so far as to claim that climate science exchanges between public agencies (both subject to Freedom of Information statutes) are too secret to be seen by this Court. This claim has no credibility, and I invite the Court to draw an adverse inference from NIWA’s obstructive behaviour. The obvious inference is that the BoM found NIWA’s unprecedented methodology to be fatally flawed. As the defects remain undisclosed, it would be dangerous for this Court to accept NIWA’s unsupported opinions on any of the scientific matters in dispute.
[The extract shown above] says that NIWA will “rely on evidence prepared specifically for the purpose of this litigation”, presumably referring to the affidavit of Kevin Trenberth. It is notable that Dr Trenberth makes no attempt whatever to analyse or justify any of the Review Adjustments. NIWA’s failure to obtain validation from any external source reinforces the adverse inference already mentioned.
This was a very different outcome than was promised by Minister Mapp. Now the Review outcomes are no more than NIWA’s in-house opinion. The Review techniques were self-invented, and have no precedent anywhere in the world. BoM apparently didn’t like them. Trenberth wasn’t prepared to vouch for NIWA’s detailed methods or results. No other expert witnesses were found to support them.
Although relying upon Rhoades & Salinger (1993) (RS93) in every chapter of the Review, the Wratt affidavit finally admits that the RS93 statistical techniques were not used in the Review. The departure from RS93 is not mentioned in the 174-page Review report. There’s no explanation for abandoning the internationally-recognised methods in the Review, in the Court documents or anywhere else.
Nearly 20 months ago, the Review stated that NIWA was still in the process of calculating the confidence levels of the adjustments, as promised by the Minister in February 2010. But they were not produced for the hearing. NIWA says they will be recorded in the promised journal paper.
But there’s no journal paper. NIWA told the Court it has been too busy defending the proceedings to have time to write it. Then, of course, there is the problem of those independent peer-reviewers…
* NZ Climate Science Coalition
Views: 547
hI Richard
I’m excited that you’re getting round to the legal stuff
Can we see the Affidavits and Exhibits on line
Can we see the transcript on line
The Australian High Court does leave application transcripts on line
and I think all Courts now that they’re sound recorded should have on line transcripts
(except for Family Courts and proceedings regarding children)
Yes, Val, I bet you are! And I’m excited to bring it to you!
I’m right now writing a brief post announcing that permission to view the documents has been granted, but access must wait for the decision. I’ll answer your questions in that. It’s a mixture of yes and no.
Our High Court is here http://www.hcourt.gov.au/
there’s a gallery there you can have a look around
I’ll alert Warwick too
Pingback: Breaking: Courtroom Chaos as New Zealand Skeptics Rout Government Climatists | johnosullivan
This may be of interest
http://johnosullivan.wordpress.com/2012/08/01/breaking-courtroom-chaos-as-new-zealand-skeptics-rout-government-climatists/
I was fascinated to read what he has to say. He writes well. Should be spiking the site traffic.
NIWA on its website is certainly saying
NIWA approached the Bureau of Meteorology in Australia to ask whether their climate scientists would undertake an independent external peer review of the methodology and documentation of the series, and they agreed. All documents have also been internally peer reviewed.
NIWA scientists have addressed the feedback from peer reviewers and have produced final versions of the reports for all the stations making up the seven stations series.
http://www.niwa.co.nz/our-science/climate/information-and-resources/nz-temp-record/review
and NIWA scientists have addressed the feedback from peer reviewers and have produced final versions of the reports for all the stations making up the seven stations series.
NIWA welcomed this opportunity to have independent expert review of its work by suitably qualified climate scientists.
Strange … to rely upon peer review for the website but not at Court
Note the careful wording: “NIWA scientists have addressed the feedback …..”.
Nothing here to suggest that NIWA accepted or adopted that feedback.
I also find it strange that Dr Wratt in the extract from his affidavit above purports to speak for BOM and also without any documentary evidence to that effect (well none from reading the extract) says The reason for that is not that the BOM peer review did not support NIWA’s approach to adjustments and homogenisation of the raw data in temperature records (it did) …
True. There certainly was no documentary evidence, because, as the post makes clear, NIWA explicitly withheld it to “safeguard” the precious peer review process. The court must rely on his affidavit alone. Odd, as you say, since NIWA knew that every other assertion in the hundreds of pages of assertions had to be supported by properly-produced paper.
This is amazing: no external validation (no reliance on BOM Review by NIWA); adverse inference; no support from expert witnesses. And yet Dr David Wratt still asserts:-
So if NIWA cannot rely on BOM’s work as evidential support in law, what does this say about BOM’s Review and why can’t NIWA rely on it in law?
Dr Wratt says BOM did support NIWA’s approach which is true but BOM only supports NIWA’s approach because that was the ONLY approach considered by BOM. On that approach BOM states: “…the review will constrain itself to comment on the appropriateness of the methods used”:-
So basically, NIWA realized that its approach was deficient in light of the more rigourous statistically verified approach undertaken by NZCSC, rendering BOM’s Review of no consequence.
Kudos to Bob D, Gary K (?), Barry B, 3 wise men (statisticians) and who ever else compiled the NSCSET Audit for exposing NIWA’s scientific deficiency in this case.
Absolutely. Also, Bob reminded me the other day that without Warwick Hughes the task of analysing NIWA’s adjustments was a hopeless one. He gave us a spreadsheet given to him several years before by Jim Salinger that showed the actual adjustments made to the raw readings, though it didn’t show why they were made.
That’s right. Warwick was the key to everything – without him we could never have resolved the 7SS into each station component, and worked out the individual adjustments. NIWA weren’t prepared to give away any information at all.
We now know the reason – they had no idea how Salinger arrived at each adjustment, and simply didn’t want to admit that the graph we were querying was as big a mystery to them as it was to us.
Remember the nonsense about altitude adjustments at Kelburn using Wellington Airport?
Yes. Eventually, it emerged that no adjustments anywhere in the country had been calculated on the basis of the station’s altitude. They were all made on the basis of neighbouring sites.
BOM:-
“The review does not constitute a reanalysis of the New Zealand ‘seven station’ temperature record”
NIWA:-
“The key result of the re-analysis is that…….”
BOM’s perspective being the Audit, NIWA’s being the compilation.
H/T Ken Perrott for inadvertently highlighting this inter-organizational conflict in ‘Painted into a corner?’
http://openparachute.wordpress.com/2010/12/17/painted-into-a-corner/
I note that Ken might be a little nervous pre NZCSET v NIWA judgment in view of what he wrote about the Review and AWFWY without the knowledge (I assume) that the issue would go to court:-
Those words just might come back to bite Ken very painfully
Richard don’t know if that letter formed part of NIWA’s case, did it?
and where’s the audit – can you provide a link? Maybe at the next post so all links to the Coalitions docs are on the same page
Val, it certainly provided some fuel, though I can’t quote specific statements we made. It must have helped counsel to conclude the BoM gave the review no support.
I see you’ve found the audit at the CSC site. It’s also available here.
thanks Richard
The BoM does not enjoy a pristine reputation among many Australian skeptics. It seems likely to me that NIWA was happy to agree to review by BoM in the belief that they could expect to receive active support from that quarter. A pudding that is too rich for BoM might be very rich indeed.
I’ve had a nagging thought that there’s a loose end somewhere in the lead up to the “Staistical Audit’ so I went back through my email “NZT7” archive and sure enough, I found this at the start:-
————————————————————————————————————————————-
What I was presenting was a simple check-sum for a cumulative calculation in a Masterton case study. As any engineer, surveyor, construction technician or drafty knows, cumulative calcs can lead to some embarrassment come construction time.
Things got rolling but after Bob came back with “I’ve been thinking about this all afternoon” everything went ballistic culminating in the ‘Statistical Audit’ but I never really got a conclusive answer except for Bob’s “From my initial reading of the Masterton data, it seems that you are right in one sense, and not so much in another”
But now with the ‘Statistical Audit’ I’ve received the most comprehensive answer to a question that has probably ever been offered to any question ever asked.
There in the Masterton section on page 22 Table 3: Comparison between NIWA and R&S results I see Site 4 Waingawa (2473) cumulative R&S sum 0.00 so my question is answered and I have a comparison for my check-sum:-
-0.55 C NIWA Review
-0.17 C Check-sum
0.00 C R&S by Bob, Gary et al, ‘Staistical Audit’
I think the check-sum is closer to R&S (0.17 not being significant IMO) than NIWA because the first and last (that being the reference) Masterton stations are non-UHI, non-altitude distorted. The same check-sum would not work if they were I don’t think.
If NIWA had calculated a check-sum for Masterton, they would have realized there was something wrong with their cumulative sum.
Amusing I think, that the original blog thread and email exchange (51 emails in that first exchange alone and as many more subsequently just in my NZT7 archive) was provoked under the post:-
Perrott puts his foot in his mouth
Richard Treadgold | February 10, 2011
Then Renowden joins him
What idealogues like Perrot and Renowden don’t realize is that sceptics hone their case to ever greater fineness by the thought provocation that erroneous commentary such as Perrott’s produces.
Absolutely correct, Richard. It was the discussion with Ken and your analyses that started me looking closely at the Masterton record using RS93.
Ken, as usual, showed zero understanding of the issues. To be honest, I suspect he still doesn’t understand what’s going on, or how he prompted us. Gareth is the same, but isn’t a scientist, so I suppose he has that excuse. I laughed when I saw Gareth had given a summary of the court case.
So getting back to motivation, thanks Ken, you were the catalyst that really got the Audit and court case moving.
To be very clear Bob, my analysis ceased about the time you started applying the R&S method as per:-
Statistical Audit of the NIWA 7-Station Review
Page 30 – 31
Appendix A
The Rhoades & Salinger (1993) Method
Description of the method
Located here for anyone interested:-
https://www.climateconversation.org.nz/docs/Statistical%20Audit%20of%20the%20NIWA%207-Station%20Review%20Aug%202011.pdf
One of my “to do’s” is to get a better understanding of that process (it’s certainly clear enough) but I confess that at present I haven’t got down to the nitty gritty,.but I’ll keep at it from time to time.
Having worked with professional engineers, architects, law practitioners and such like, I learnt to recognize the boundary of my technical ability and from then on to defer to relevant expertise. I think perhaps NIWA (and climate science generally) should learn to do the same.
BTW, made an interesting discovery recently. Anyone in New Zealand can provide legal information as long as they don’t encroach upon the “reserved areas of law” as described in s6 of the Lawyers and Conveyancers Act 2006 (LCA), which covers the provision of advice in relation to the direction or management of legal proceedings. and don’t hold themselves out as something they are not e.g. qualified in law. I’ve always wondered what the limits were because I’ve been in the position to advise on council plan changes (and have done so by contract) but never knew the bounds in respect to Acts and Regulations even though I’ve had to have a working knowledge of several – now I do.
Requirements for offering legal services on the internet
http://my.lawsociety.org.nz/in_practice/practising_law/regulatory_advice/requirements_for_offering_legal_services_on_the_internet2
This could be useful in regard to the RMA (and perhaps the ETS).where in the RMA, conflicts can be resolved and agreements reached via mediation and independent hearing without needing to go to court.
An Everyday Guide to the Resource Management Act
http://www.mfe.govt.nz/publications/rma/everyday/
“NIWA’s minister (the Hon. Dr Wayne Mapp) added three further undertakings”
#3 by June 2011, a scientific paper describing NIWA’s methods would be submitted to an academic journal for publication, and would be subject to independent peer reviews.
What is required here is the equivalent to BOM’s paper (although not a peer-reviewed journal publication):-
Trewin, B. 2012.
Techniques involved in developing the Australian Climate Observations Reference Network � Surface Air Temperature (ACORN-SAT) dataset.
CAWCR Technical Report No. 049
http://www.cawcr.gov.au/publications/technicalreports/CTR_049.pdf
Note that BOM goes a step further than NIWA and adjusts out “weather dependent” inhomogeneities (“break points”). Apparently weather events and seasonal variations do not a temperature record make.
All CAWCR Technical Reports here:-
http://www.cawcr.gov.au/publications/technicalreports.php
But, but… it’s a record of the temperature! I’ve not heard of this. Why is a weather-caused step in the temperature record not part of that very temperature record?
“Why is a weather-caused step in the temperature record not part of that very temperature record?”
From Trewin 049 page 96 pdf:-
All over the world, rain makes grass grow, so naturally wind speed, near surface mixing and therefore temperature are modified but that is no reason to adjust out the change, rather it should be recorded as such because the record is of weather, seasons and climate IMO.
This is quite different to say Albert Park where, as I understand, trees grew but they kept on growing. The Alice Springs grass on the other hand, grows when it rains, dries up when it doesn’t rain, grows again when it does and so on, that’s the nature of weather and grass – ask our dairy farmers about that.
I would add, to put the ACORN weather adjustments in perspective:-
The 1st +0.8 C Alice Springs Min step at 2004/05 is due to WEATHER (seasonal vegetation change)
The 2nd -0.3 C Alice Springs Min step at 2001/02 is due to SITE MOVE (“a few metres west”).
The NZCSET v NIWA case is essentially about (among other things) site move adjustments (as in the 2nd step above) but in the Australian situation, the site move adjustments can be secondary in magnitude to the weather adjustments (nearly 3x in that Alice Springs case).
Even then, the -0.3 C site move step is at the lower limit of BOM’s ACORN step adjustments.
They do NOT adjust for a step (break point) LESS than 0.3. Prior to that in the HQ series they had made 0.1 C adjustments.
So if (hypothetically) the 2nd Alice Springs Min site move step at 2001/02 had worked out at -0.29 C, they would not have made an adjustment and the only adjustment this century would have been the weather adjustment at 2004/05.
I’ve just counted 8 R&S adjustments in the ‘Statistical Audit’ that were less than 0.3 C.
Using BOM’s ACORN-SAT criteria, those adjustments would NOT have been made e.g. there would be NO adjustments to Auckland (0.00, +0,02, -0.12, 0.00, R&S cumulative sum for Auckland -0.1, NIWA -0.62).
That is, BOM ACORN-SAT criteria makes even less site move step adjustments (after R&S application) than the R&S criteria does.
That’s good stuff Richard. Interesting.
Richard … just in respect to what you say NIWA’s minister (the Hon. Dr Wayne Mapp) added three further undertakings ….
I recall there was an excerpt from (your Hansard) but I can’t locate it
Can you? If so could you please post a link
Truly magnificent work, and kudos to all. I have been peripherally watching the progress of this litigation, and the result thus far is incredible.
Shine a torch, and watch the roaches run for cover……