Grady on the Conference Board and TILMA

Patrick Grady, a former senior Finance official and leading mainstream economist, has weighed in on the Conference Board’s estimate of TILMA’s economic benefits. He cites the paper that Marc and I wrote and reiterates the points first made on this blog. He also notes that the Conference Board’s own forecast of BC’s economic-growth rate does not seem to reflect its projection of TILMA’s effect.

UPDATE (April 4): Yesterday, I mentioned this item to Larry Hubich, President of the Saskatchewan Federation of Labour. Today, he has it in the province’s two main newspapers: The Leader-Post and The StarPhoenix.

Patrick Grady
The Conference Board of Canada’s $4.8 Billion Estimate of the Impact of the BC-Alberta TILMA is not Credible
April 2, 2007

The Conference Board of Canada prepared “An Impact Assessment of the BC/Alberta Trade, Investment and Labour Mobility Agreement” for the British Columbia Ministry of Economic Development in September 2005. More specifically, it estimated that the TILMA agreement would increase BC GDP by 3.8 per cent or $4.8 billion and create 78,000 jobs (based on the assumption that the average employee in BC produced $62,091 worth of goods and/or services in 2004).

Coming from such a prestigious organization as the Conference Board, this estimate has received much publicity and has contributed to a widespread perception that there are huge costs associated with existing barriers to interprovincial trade. While removing barriers to interprovincial trade is certainly good policy, it is also important that any such decisions be based on sound empirical analysis otherwise disappointment will be the inevitable result.

The first thing about the Conference Board estimate that raises questions is that it is so much higher than previous estimates which were generally less than 0.5 per cent of GDP for the elimination of barriers across the whole country. It should follow that the estimated gains for removing barriers between two provinces would be less than the benefits of removing them between all ten and the three territorial governments. In addition, many of the earlier estimates of the impact of scrapping barriers were made before 1994 when the Agreement on Internal Trade took effect and lowered barriers. If those estimates were to be redone now using the same methodology, they would be even lower.

The second thing that should raise flags is that an increase of 3.8 per cent of BC’s GDP is very large in relation to the 5.4 per cent share of BC’s GDP that went to Alberta in interprovincial exports in 2003 the latest year for which Statistics Canada data is available. Yet the barriers being eliminated are relatively minor.

The third thing is that the Conference Board does not seem to have put much weight on its own estimate in preparing its economic forecast for BC. In its Provincial Outlook, Winter 2007 the Conference Board forecasts real GDP in British Columbia will advance by 3.1 per cent in 2007 and 3.3 per cent in 2008, which is only slightly higher than total Canadian growth of 2.7 per cent forecast for 2007 and the same as that forecast for 2008. In its Provincial Outlook Long-Term Economic Forecast: 2007 the Conference Board projected the same 2.2-per-cent long term growth for British Columbia and the Canada out to 2030. If the Conference Board really believed that the TILMA was going to raise BC GDP by 4.8 per cent, it should have forecast a relatively higher growth rate for BC at some point in the future.

To understand exactly why the Conference Board’s TILMA estimate is not credible, it is necessary to examine its methodology. In contrast to most of the other earlier estimates of the impact of removing barriers which sought to quantify specific barriers and estimate their economic impact, the Conference Board made use of a survey that asked respondents to rank the impact of the TILMA on eleven industries (agriculture; primary, utilities; construction; manufacturing; wholesale and retail trade, transportation and communication; finance, insurance and real estate; commercial; non-commercial; and public administration) and on seven regions (Vancouver Island and Coast; Lower Mainland and Southwest; Thompson-Okanagan; Kootenay; Cariboo; North Coast and Nechako; and Northeast).

The ranking was one on a seven-point scale ranging from -3 to +3 with -3 indicating a significant challenge (a negative impact greater than 10 per cent of GDP), -2 a moderate challenge (a negative impact between 5 and 10 per cent of GDP), -1 a small challenge (a negative impact between 0 and 5 per cent of GDP), 0 no impact or not applicable, +1 a small benefit (a positive impact between 0 and 5 per cent of GDP), +2 a moderate benefit (a positive impact between 5 and 10 per cent of GDP), and +3 a significant challenge (a positive impact greater than 10 per cent of GDP).

The Conference Board sent out the survey to 24 organizations, 11 from government ministries and 13 from industry organizations, but only got responses from 10: 6 from government ministries, and 4 from the private sector. And 3 of those that did respond provided no regional detail and 2 no industrial. Needless to say, this small response particularly from the private sector undermines the reliability of the information collected because it means that most of the respondents are likely to have no specific industrial or regional expertise to contribute to the estimate. In addition, it’s not clear from the documentation that the respondents were actually informed of the intended relationship between the ranking and the numerical score later assigned by the Conference Board.

The Conference Board used the information from the survey in combination with its own judgment to produce its estimate. It did this by putting a numerical estimate for the impact on each industry and region based on its -3 to +3 scale and then calculated weighted regional scores using employment weights from the labour force survey for the appropriate industry and region. No list of barriers was provided and no empirical studies of the impact of barriers were cited to justify the scoring.

All of the scores assigned by the Conference Board were less than 1 and the overall score for BC rounded to 0.76. The Conference Board then applied this to the five percent top of the 0-to-5-per-cent small benefit scale yielding the widely publicized overall estimate for BC of 3.8 per cent of GDP or $4.8 billion. The Conference Board didn’t make clear in its documentation why it shifted from interpreting the 1 ranking as applying to the 0-to-5-per-cent range to interpreting it as applying to the top of the range. If score was based solely on the survey responses, this would not be appropriate as the most reasonable assumption would be to assume a uniform distribution within the range. In that case, the score would be applied to the 2.5-per-cent midpoint of the range. But since the Conference Board transformed the whole thing into a judgmental exercise with survey responses only being used as input and since the scores they used were lower than the weighted survey responses, it is possible that they might have made some sort of adjustment.1

But regardless of how you look at it, the Conference Board estimate appears implausibly high in relation to previous estimates and in relation to BC’s exports to Alberta. Again, the Conference Board must have even thought so themselves as it seemed to have disregarded its estimate of the impact of the TILMA in preparing its latest forecast for the BC economy. And, if even the Conference Board doesn’t believe its own estimate of the impact of the TILMA on BC, those who continue to cite this estimate as evidence of the large magnitude of the benefits to be derived for British Columbia from the TILMA don’t really have a leg to stand on.

Notes

1. In a less charitable interpretation of the Conference Board’s calculation, Erin Weir and Marc Lee (2007, p. 1) allege that “After arbitrarily ‘scoring’ BC’s regions and industries, the Conference Board doubles its estimate of TILMA’s benefits through a simple arithmetic error.” The error is applying the 0.76 to 5 per cent instead of 2.5 per cent the midpoint of the range. I prefer to consider this an “adjustment” in going from the survey to the judgemental estimate rather than an “arithmetic error.” Nevertheless, our criticisms of the magnitude of the Conference Board estimate ends up being pretty much the same.

References

Conference Board of Canada (2005) “An Impact Assessment of the BC/Alberta Trade, Investment and Labour Mobility Agreement,” Prepared for the British Columbia Ministry of Economic Development, September.

______ (2007a) Provincial Outlook, Winter 2007.

______ (2007b) Provincial Outlook Long-Term Economic Forecast: 2007.

Erin Weir and Marc Lee (2007) “The Myth of Interprovincial Trade Barriers and TILMA’s Alleged Economic Benefits,” Canadian Labour Congress, Research Paper No. 43, February 26.