Employment Data: Working on a Mystery
This blog flagged, and Worthwhile Canadian Initiative pursued, a striking discrepancy in July’s employment data. The Survey of Employment, Payrolls and Hours (SEPH) indicated that employers paid 74,000 more employees.
Conversely, the Labour Force Survey (LFS) had indicated that employers paid 79,000 fewer employees in July. This difference of 153,000 exceeds 1% of Canada’s workforce.
Today’s release of SEPH figures for August sheds more light on this mystery. These figures show a loss of 110,000 jobs. LFS had shown a gain of 38,000 paid positions in that same month.
While these numbers imply another large discrepancy (148,000), it almost exactly offsets the previous one. Combining July and August, SEPH indicates a decrease of 36,000 and LFS indicates a very similar decrease of 41,000.
Statistics Canada addresses this issue specifically in today’s SEPH release. It helpfully notes important differences with LFS and points out that the two sources identify congruent trends over several months, if not in each individual month.
The basic message, I think, is that survey data bounce around a bit from one month to the next. So, we should not be shocked when the two surveys bounce in opposite directions in any particular month.
The problem is that this dynamic aggravates other limitations in Canada’s labour-market data. Earlier this week, The Globe and Mail noted the lack of current data on exhaustion of Employment Insurance benefits.
When the Employment Insurance rolls shrink in a particular month, checking whether employment rose or fell that month provides at least some indication of whether people left their benefit claims because they found work or ran out of benefits without finding jobs.
However, if SEPH and LFS provide mixed signals for a given month, then even such back-of-envelope analysis may be impossible. (In practice, I mostly just use LFS because SEPH comes out after the Employment Insurance figures.)
I was talking to someone from StatsCan the other day, and it turns out that even after you correct for the fact that LFS has self-employed workers and SEPH doesn’t, you *still* get big discrepancies. And in answer to my question about which to use, he could only shrug: he couldn’t say which is a better indicator.
The LFS figures above are on employees (i.e. excluding the self-employed).
I already went through quite through the differences in the two surveys. If you want further explanation email me.
Bottomoline is, LFS is an establishment based survey that measures payroll data from the payroll admin file. It is then basically converted into what they call “payroll” employment. This calculated by taking total payroll for a specific geographic area and dividing by the factor that approximates average wage for that area- you are left with an estimate of total payroll jobs.
It can be effected very much by overtime, bonuses, strikes, major layoffs and such. But these are typically dealt with through ARIMA II seasonal adjustments and other adjustments as deemed necessary. If you get some stable streams in the payroll form month to month you get a pretty clear cross sectional snap shot of employment. Of course these are only as good as the quality of the data in the admin file.
LFS is a household survey and has a sample size that is quite robust for the Canada level for estimating total employment and unemployment. However it is not necessarily designed to produce changes in the two to the accuracy and reliabilty levels that many I am sure would like, and therefor the variance in the month to month. And that is after some quite fancy rotational sampling that in some ways emulates a longitudinal design taht provides for smoother month to month changes, but at the cost of less reliability.
Plain and simple here is the problem, more sample for LFS produces more accuracy, a cleaner admin file which means money results in better quality data for SEPH.
Bottomline- if you want better numbers- the surveys need more money. If you go through the recent history, you will see budgets have been cut. So the quality comes down and the variance between the two grows.
Yes they do also measure different populations, SEPH has not self employed because the payroll file for such does not exist. I also believe there is a difference in armed forces, inmates and farmers, but the biggest difference is self employed. SEPH also suffers from double counting of jobs, but again it is not something that will make up such differences.
Anyway this is all public somewhere scattered throughout the agencies public archives. Nothing secret here.
Sorry the third line should say,
Bottomline is, SEPH is an establishment based survey. (that was not the spot make a mistake huh)
Comment for Stephen on which is better to use.
All depends on what you would like to look at.
If you are digging far into the geography or some other domain, I would use SEPH as it is admin data based and as long at the payrolls are subdivided appropriately at the head office for the enterprise, a good lot of the geographic profiles will be supported. LFS you cannot go very far into many sub-domains as the CVs start jumping fairly quickly and quality fall off sharply.
If you have a particular study you are contemplating email me and I might be able to help. I worked on parts of both for a few years.
LFS is preferred by many users because it provides information on age, sex, and occupation, as well as industry, and now has detail on Aboriginals off-reserve and immigrants. SEPH is simply a payroll count by industry.
I am not aware of any sub-provincial breakdowns of SEPH data being published, but I suspect that any geographic breakdowns, even just to the level of the provinces, are of unknown quality because the underlying payroll information from business will be subject to varying treatment from one company to the next.
In past comparisons I found that even for annual data the two surveys not only gave different figures on annual change, they sometimes differed in direction.