The Ontario Auditor’s damning report on P3s

The Ontario Auditor General’s 2014 Report includes a chapter on Infrastructure Ontario’s P3 program that is particularly damning–and corresponds with many of the criticisms made on this blog and elsewhere by myself and others.

While the headlines were that P3 projects cost the province an additional $8 billion than if they were procured traditionally, the report documents numerous other problems with the province’s P3 program and practices that should appal anyone concerned about responsible public policy and the appropriate use of public funds.

The problems identified by the Ontario AG with these P3 (or AFP “Alternative Financing and Procurement” as Ontario calls them) projects aren’t unique to Infrastructure Ontario.  In fact they are common to the “Canadian P3 model” and are undoubtedly worse in terms of the practices of other provinces and the federal government –and unfortunately for municipalities, First Nations and other public entities who are having P3s being foisted upon them by federal and provincial governments.

Given how much money is being steered through P3s, other auditors must take serious consideration of these criticisms and follow up with similar audits themselves.   While each province may have their own P3 agency and may have specific requirements, from what I’ve seen, most involve business cases and value for money assessments that resemble the Ontario model.

What’s just as important is that governments provide full transparency of the costing and other details of P3s, particularly all the details that go into the key Value for Money (VfM) assessments (and especially the details on assumed risk transfer) that are used to justify P3s.

The Value for Money assessments and business cases made public are little more than window dressing justifications for decisions that have already been made.  I and others have requested background information or even their models, but P3 agencies have refused to make further details available claiming that they are protecting “commercial confidentiality”, which is a thoroughly specious excuse.  In reality, as we’ve long suspected and as this report by the Ontario AG demonstrates, all they are protecting by keeping this information secret are their own absurd assumptions.

Here’s a summary, with some commentary:

The real tangible costs for Infrastructure Ontario’s 74 P3 projects were estimated to be $8 billion higher than if the projects were contracted out, financed and managed by the public sector using traditional procurement (but private construction of course).

All of the projects were justified as P3s on the basis that they transferred large amounts of risk to the private sector.  The risks retained through public sector procurement were estimated to be about five times higher ($18.6 billion) than if they proceeded as P3s ($4 billion): with an assumed $14.6 billion transfer of risk through P3s.   While the Ontario AG report doesn’t include this information, the risks assumed transferred through Ontario’s P3s have averaged about 50% of their capital cost, with some close to 100% of their capital costs!

However, the report notes (p. 203) “there is no empirical data supporting the key assumptions used by Infrastructure Ontario to assign costs to specific risks”  and “In our discussions with the external advisors (e.g. consultants such as Deloitte etc.) , they confirmed that the probabilities and cost impacts are not based on any empirical data that supports the valuation of the risks, but rather on their professional judgement and experience.”

Yes, that’s right, Canada’s largest P3 Agency makes decisions on tens of billions of dollars of public spending using assumptions with no basis in fact, assumptions about risk transfer that are pulled out of the air, and simply fabricated by external consultants (who also happen to be well remunerated for these activities).

This was already clear from the only documents they released publicly on their Value for Money methodology including their Risk Assessment Templates.   These short documents just provides a summary guide of their risk matrices, but it’s clear these assumptions of risk transfer are not based on any empirical evidence, e.g. they come from a fact-free zone.   We’ve come to expect this type of evidence-free policy-making from the Harper government, but I think most of us expect more from our provincial governments.   And as a resident of Ontario and former Ontario public servant (at Finance), I am profoundly embarrassed by this type of policy making, as others should be.

The liabilities and commitments that the Ontario government still has outstanding relating to these 74 P3 projects amounted to an estimated $28.5 billion at the end of the last fiscal year (March 31, 2014).   The actual amount that the Ontario government has steered into P3s is more than this, as this amount doesn’t include payments already made.

The AG report highlights just two of the 90 “risks” used, but they alone were used to justify a supposed $6 billion in risk transferred, and states that they are clearly inappropriate.   One of these risks is related to “asset residual value”, with the assumption that the asset would be in better condition at the end of the contract as a P3 than if it were publicly managed, even though all costings for both the P3 and the traditional procurement options include considerable funds for major maintenance and repair.   Not only is this double-counting but it’s also a very questionable assumption: that a private sector operator will build and keep a facility that it is only responsible for a limited time period in better shape than the public sector.  Yes, there’s a problem with the public sector not spending $$ for maintenance, but in general public facilities are built to last longer than the private sector.  Another $3 billion in risk associated with “planning, process and allocation” practices has been assumed transferred through P3s, but as the AG report notes these risks are common to both procurement methods.

These are just two of the 90 or so types of “risk” that are calculated (well, really made up) and assumed transferred to the private sector in order to justify P3s.  The reality, as others have also noted, is that risks are rarely transferred to the private sector through P3s and the ultimate risk for the project and for delivering the public services always rests with the government.   The private consortiums that operate P3s projects are always constructed as separate SPVs (Special Purpose Vehicles) which means the larger companies behind them can walk away from the project at any time, risking just the equity they’ve put into the project, which is typically just 10-15% of the project cost.   Any assumption of risk transfer in excess of the actual private equity remaining in a project has no basis in financial reality.

The government/public entity will remain liable to pay out the bond holders at their high rates of interest for decades, as we found with the $1 billion Ontario gas plant scandal, and so the risks involved in P3s are often higher than through traditional public financing and procurement.

The AG report also notes that Infrastructure Ontario’s estimated costs for projects were on average 27% higher than the actual contracted costs.  This is quite extraordinary.   P3s have also been justified on the basis that they impose a stronger discipline on developing detailed costing, but what we see from this is that their estimates are way off.    My suspicion is that Infrastructure Ontario and other P3 agencies inflate their cost estimates so they can be assured of getting bids within these levels and so they can also later claim that they deliver these projects on budget.  Clearly it’s easy to deliver something on budget if you’ve already exaggerated the budgeted costs by an average of almost 30%!    What should be as much of a concern is that these inflated estimated costs mean that the P3 Agencies aren’t driving much of a bargain with P3s, by indicating that they are prepared to pay much more for the projects.

The AG report is also critical of Infrastructure Ontario’s practice of adding additional fictitious costs to their estimated  costs for traditional public procurement under the justification of “competitive neutrality”.   These types of fictitious costs include calculations of the amounts for taxation (even though many P3 corps seem to be based in tax havens) and for insurance, etc. on the rationale that the “public sector bid” should include these costs to make the playing field level.  But they aren’t real costs that would be incurred by the public entity.  There are numerous problems with using this practice, not just with P3s, but with other calculations of the comparative cost of contracting out vs in-house service.  Why should the public sector be penalized if it can provide a service more efficiently (such as through self-insurance) than the private sector?   What about broader public and social benefits?  Other measures bonus the private sector for its supposed benefits, so why treat the public sector differently?     What this shows is that the $8 billion difference between public and P3 procurement for these projects highlighted by the AG report is a low estimate of the actual difference, because Infrastructure Ontario has already added addition fictitious costs to their “public sector comparator”.

The report also notes that there’s very little competition among the large P3 general contractors, with five of them being awarded over 80% of the projects. and just two of the facility management companies being awarded a majority of the P3 projects with a maintenance component.   From my observation, if there’s any competition here, it’s all pretty chummy, with the different companies carving up their pieces of the P3 cake in a pretty friendly manner.

Also disturbing is the fact that Infrastructure Ontario was unable to provide the AG with signed conflict of interest declarations or disclosures of relationships for those evaluating submissions for a number of projects.   There seems to be a bit of a revolving door and a close relationship between people working in the financial industry on P3 and these P3 Agencies, so not having these available should be a big concern (though not a surprise).

Another major concern (but not identified by the AG report) is that many P3 agencies are charged with both promoting and assessing P3s, which is a massive conflict of interest in the agencies’ missions in terms of public policy and one that can only be addressed by separating these roles.    Given this, it’s no wonder that, as the AG report notes on p.202:

“None of these (Infrastructure Ontario’s 200) VFM (Value for Money) assessments has shown a negative VFM from using the AFP model.  In other words, all of the VFM assessments concluded that the delivery of the projects would be cheaper under the AFP approach than the public sector. The assessments are accompanied by a letter from an accounting firm that acknowledges that the assessment was prepared in accordance with Infrastructure Ontario’s methodology.  However, all letters contain a disclaimer by the firm that it has not audited or attempted to independently verify the accuracy or completeness of the information used in the calculation of the VFM.”

If that isn’t enough, there’s the example of an Ontario college ( p. 199, 208-9) with a project that involved a building with a classroom and retail space built through traditional procurement in Phase 1 that was then directed to procure Phase 2 of the project, the construction of a very similar building, as a P3.  After inflation and adjustments for building differences the Ontario AG estimated that the P3 cost an estimated 10% more per square foot, but it demonstrated a positive VFM as a P3.  While the college and mayor of the city (presumably former Mississauga Mayor “Hurricane” Hazel McCallion) tried to get the college released, the Ontario insisted it be done as a more expensive P3.

Reading this, it’s very hard not to get the sense that they are cooking the books to always make the AFP P3 alternative always come out as the superior option through these VFM assessments.   And given the fact that Infrastructure Ontario refuses to make public the more detailed cost calculations from any specific Value for Money assessments, or to even release the more detailed generic risk assessment matrices and assumptions they use makes anything associated with these P3s extremely suspicious indeed.   Really–what are they afraid of?

Once again, the problems the Ontario AG identified are not unique to Ontario.   The P3 Business Cases and Value for Money Assessments (where you can get them: many jurisdictions don’t release them and those that do redact them very heavily) that are used elsewhere in Canada are very similar to those used in Ontario and are prepared by the same accounting firms.

There are other disturbing concerns raised in the AG report that I won’t get into here, except for a final one.

Apparently the range of creative accounting and double-counting methods used by Infrastructure Ontario to justify its P3 projects isn’t enough already.   The AG reports that the agency was planning to mak a number of changes to its methodology, including adding an “innovation adjustment” of up to 13.3% to the base cost on the public sector comparator side (in other words, making the cost of the traditional public procurement option appear up to 13.3% more expensive) on the justification that private sector bidders were containing costs that they hadn’t anticipated.   But as the report gently states (p. 205):

“However the average bid coming in below budget could also be due to a number of other factors, such as the overly generous budget estimates and changing market conditions, and may not necessarily be related to innovation.” 

The Ontario Auditor General report makes a number of recommendations that Infrastructure Ontario has responded fairly weakly to, while the defensive reaction of the Premier and Ministers on this is also disturbing.     While the report is damning already, in my view it could and should have gone further in calling for much greater reform as well as transparency and accountability.   We shouldn’t have to wait for the province to waste another $8 billion dollars, let alone $1 million, on expensive P3s, while it cuts public spending and services in other areas.

The stink coming from this is so bad that any responsible government would announce an immediate moratorium on P3 projects pending a thorough reform of Infrastructure Ontario and the province’s P3 policy and processes to introduce more accountability, as Manitoba has done.  And the province should require the public release of the un-redacted business plans and complete details behind the value for money assessments for every project (as well as Infrastructure Ontario’s complete methodology) to be publicly released as a matter of course.   Any and all P3s will–and should–be under a cloud of suspicion until and unless this is done.