When Good Data Goes Bad: The NHS2011

This piece was  published today in the Globe and Mail’s Economy Lab. 

Two findings stand out in the National Household Survey (NHS) data released Wednesday, both critical in this post-recession era of uncertainty:

1) A quarter of Canadian households spent 30 per cent or more of their pre-tax income on shelter, the official measure of housing affordability.

2) There was virtually no change in the rate of home ownership between 2006 and 2011. It was 68.4 per cent in 2006, and 69 per cent in 2011.

I’ll explain the significance of both; but first, let’s consider how this information was collected.

The federal Conservative government spent an extra $22-million* to replace the long-form census questionnaire with the survey, going from a 20-per-cent sample of households receiving the long form in which responding was mandatory, to a 30-per-cent sample of households who could answer the questions if they felt like it. Why? The feds insisted the census was a coercive instrument of government that violated of your privacy. (I know. See my blogs from summer and fall 2010 here. Statistics Canada has a peerless record in assuring complete confidentiality in both voluntary and mandatory information-collecting.)

Of course, non-response rates are higher with voluntary surveys, and various studies show there is a predictable bias to those who are less likely to respond: The rich and the poor; those who are not fluent in either official language; Aboriginal populations; and people with disabilities.

In places where more than half the sample didn’t respond, StatCansuppressed the findings. Fewer than 60 per cent of Saskatchewan’s census subdivisions had useable information.

For these reasons, Statistics Canada made clear that you can’t compare NHS findings to previous census or other surveys they have conducted. The NHS is a family portrait with a lot of people missing.

That means we can’t ask the important questions about the differences in how we fared before and after the 2008 crisis on critical issues such as housing affordability. We also don’t really know how big our problems are, and to what degree they are widespread or localized.

So when we read that a quarter of Canadian households are paying more than 30 per cent of their pre-tax incomes on shelter, we know it’s a conservative estimate (if you’ll pardon the pun) because of the higher non-response rates among low-income households. When we read that twice the proportion of renters than owners are paying more than 30 per cent of their pre-tax incomes on shelter (40 per cent of renters versus 18 per cent of owners), we know that this understates what’s going on, because more renters are low-income than home-owners.

And how many Canadian households are renting? It’s not clear that we even have that basic question right. Statistics Canada notes some unexpected results with the NHS. It should be the inverse of the 69 per cent of owners – that is, 31 per cent of Canadian households rent their dwelling. But the proportions may be off.

It’s odd that the NHS shows the rate of growth in home ownership fell precisely when interest and mortgage rates dropped to historic lows between 2006 and 2011. (After all, interest rates weren’t this low even in the 1930s, and mortgage rates are lower than at any time since the 1950s, when authorities started collecting the data.)

Home ownership is the way many people create economic security and maintain purchasing power when incomes fluctuate. Tracking that variable authoritatively can tell us whether troubling levels of household debt are being driven by a rising incidence of home ownership (perhaps not such a bad thing), higher housing prices (bigger mortgages), or other causes. The census did that.

Census results also alerted us to systemic over-crowding in First Nations and Inuit housing. The NHS slapped blurry goggles on our vision. We know less than before about the living conditions of these communities now. Same story for immigrant and newcomer communities.

These are the two nodes of future population growth in Canada – where the evidence we do have suggests that troubles are brewing.

The main finding of the NHS is that, with weaker ability to track change or measure a growing portion of society, we’re losing sight of the Canada we’re becoming.

 

* Note: An earlier version of this piece said the National Household Survey cost an additional $30-million over the cost of the long-form census that it replaced. It was budgeted to cost that much more, but Statistics Canada has indicated that the additional cost was only $22-million.