Review of studies comparing Canadian and US health outcomes

This was referred to by Ezra Klein at the very bottom of the last post, and it deserves its own. Abstract and the concluding “discussion” section are pasted below:

A systematic review of studies comparing health outcomes in Canada and the United States

 

Gordon H. Guyatt, P.J. Devereaux, Joel Lexchin, Samuel B. Stone, Armine Yalnizyan, David Himmelstein, Steffie Woolhandler, Qi Zhou, Laurie J. Goldsmith, Deborah J. Cook, Ted Haines, Christina Lacchetti, John N. Lavis, Terrence Sullivan, Ed Mills, Shelley Kraus, Neera Bhatnagar

 

ABSTRACT

Background: Differences in medical care in the United States compared with Canada, including greater reliance on private funding and for-profit delivery, as well as markedly higher expenditures, may result in different health outcomes.

Objectives: To systematically review studies comparing health outcomes in the United States and Canada among patients treated for similar underlying medical conditions.

Methods: We identified studies comparing health outcomes of patients in Canada and the United States by searching multiple bibliographic databases and resources. We masked study results before determining study eligibility. We abstracted study characteristics, including methodological quality and generalizability.

Results: We identified 38 studies comparing populations of patients in Canada and the United States. Studies addressed diverse problems, including cancer, coronary artery disease, chronic medical illnesses and surgical procedures. Of 10 studies that included extensive statistical adjustment and enrolled broad populations, 5 favoured Canada, 2 favoured the United States, and 3 showed equivalent or mixed results. Of 28 studies that failed one of these criteria, 9 favoured Canada, 3 favoured the United States, and 16 showed equivalent or mixed results. Overall, results for mortality favoured Canada (relative risk 0.95, 95% confidence interval 0.92–0.98, p = 0.002) but were very heterogeneous, and we failed to find convincing explanations for this heterogeneity. The only condition in which results consistently favoured one country was end-stage renal disease, in which Canadian patients fared better.

Interpretation: Available studies suggest that health outcomes may be superior in patients cared for in Canada versus the United States, but differences are not consistent.

 

Discussion

In this systematic review, we demonstrated that although Canadian outcomes were more often superior to US outcomes than the reverse, neither the United States nor Canada can claim hegemony in terms of quality of medical care and the resultant patient-important outcomes. In virtually all areas, study results have demonstrated some apparent advantages for Canada and others for the United States. In cancer, where a number of strong studies have used population-based registries, Canadian outcomes appear superior in head and neck cancer, and possibly for low-income patients with a variety of cancers; American women with breast cancer appear to have better survival rates than Canadian women. In data from population-based registries, Canadians enjoy better risk-adjusted survival after a variety of surgeries, but American outcomes appear superior after hip fracture repair and cataract surgery. Studies that do not utilize population-based registries suggest that Americans have, possibly as a result of more aggressive interventions, less angina after MI, but the benefit may come at the price of increased strokes and bleeding. There is one area in which Canadian outcomes appear consistently superior: end-stage renal failure. Even here, however, as we shall discuss, one cannot be certain that superior medical care is responsible for the differences.

The strengths and limitations of this systematic review bear on its interpretation. We established a team that included expertise in medicine, clinical epidemiology, health economics, health policy, and health services research in both Canada and the United States, developed explicit eligibility criteria, and conducted a comprehensive search that uncovered a number of eligible articles not included in a previous systematic review.51 We excluded studies, such as randomized trials of medical interventions in which Canadian investigators recruited some patients and American investigators others, in which care would be idiosyncratic or atypical of care in usual clinical practice. Our thorough examination of each study addressed issues of validity (selection of populations, adjustment for confounders, loss to follow-up) and generalizability (breadth of samples, including specifying studies that came from population-based registries).

Reviewers who determined eligibility and judged validity and generalizability were blind to the results of the study. In decision-making regarding methodologic issues that arose as the review progressed, we recused investigators who were aware of the study results. We made explicit a priori hypotheses regarding possible sources of heterogeneity, and tested these hypotheses in a thorough statistical analysis. Our results are consistent with those of a prior systematic review that completed its search (less comprehensive than ours) in 1997, conducted a limited assessment of study validity, and failed to conduct a formal meta-analysis.51

The main limitation of our review is in the uneven quality of the original studies, and the threats to validity that remain even in those studies of high quality. There were two key ways a study could fail to adequately address our question: either the population might be small or narrow, or the investigators might not carry out statistical adjustment for potential differences in underlying prognosis. Most of the studies we identified failed one of these two criteria (Tables 2, 3 and 4).

Even studies that meet these criteria, and meet the more rigorous criterion of utilizing population-based registries, present challenges with respect to their interpretation. In general, a health care system can improve outcomes in two ways. One is to facilitate early entry to care, including preventive care, and thus avoid unnecessary morbidity and mortality. For instance, if access to primary care is easy and without financial obstacles, one might expect superior outcomes in hypertension (e.g., fewer strokes). Alternatively, a system might generate better outcomes by better treatment of serious morbidity once it arises. For instance, stroke patients may be more likely to receive early thrombolysis, thromboprophylaxis, and multidisciplinary rehabilitation.

If a health system does better in early identification and treatment, diseased patients in that system will appear less ill. Statistical adjustment for severity of illness is in general appropriate – one wouldn’t want to attribute to better care what is in fact due to a better prognosis. The risk, however, is that the adjustment will obscure the benefits of early identification and treatment.

Such issues become relevant in comparisons of outcomes between Canada and the United States. For instance, the United States does a better job of screening women for breast cancer.52 To the extent that early diagnosis reduces breast cancer deaths, one would expect a survival advantage for American women. At the same time, any apparent increase in longevity may be largely, or even completely, due to the length and lead-time biases inherent in observational studies of screening.

A number of studies using the American National Cancer Institute’s Surveillance, Epidemiology, and End Results Program (SEER) and the Ontario Cancer Registry (OCR) have addressed breast cancer outcomes. Although studies using these databases and examining Toronto versus a number of US cities suggest higher breast cancer survival in low-income Canadian women than in their American counterparts,20,21,36 several studies using the entire database have suggested superior overall breast cancer survival in American women.18,19,32 We rated these studies as low quality because of failure to adjust for disease stage. If higher screening rates or better self-detection in the US result in the identification of earlier stage histologic cancers that would have remained asymptomatic and dormant, studies would demonstrate superior survival despite equivalent medical care. On the other hand, perhaps there is a true American advantage that results from higher rates of screening52 or from superior care after diagnosis. The data do not allow assessment of the relative likelihood of these possible explanations.

These studies raise another important limitation of the current data. Canada has largely53 (though not completely) eliminated gradients in access to care by socioeconomic status that remain in the United States,55,56 and this may contribute to Canada’s smaller socioeconomic gradients in health outcome.57 If this were so, one would expect that studies focused on poorer individuals would reveal superior outcomes in Canada, whereas differences might be obscured in studies of entire populations. Indeed, the cancer studies by Gorey and colleagues20,21,36 and by Boyd19 suggest this may be the case. At the same time, it is possible that being able to pay for better care might lead to better outcomes in those with high incomes in the US versus Canada. Indeed one of the studies in cancer patients suggested this possibility.19 Unfortunately, these are the only studies that explore gradients in outcome across socioeconomic status.

Although the overall effect in the meta-analysis may be of some interest (a 5% reduction in relative risk of all-cause mortality in Canada versus the United States) the large variability in study results (heterogeneity p < 0.0001, I² 94%, Figure 2) makes the pooled estimate difficult to interpret. Our primary reason for conducting the statistical analysis was, through meta-regression, to explore possible explanations of variability in results and provide adjusted estimates of relative risk. This exploration proved difficult to interpret. Although the multivariate model identified apparent sources of heterogeneity and provided adjusted estimates of relative risk (Table 5), the results were inconsistent between univariate and multivariate approaches, and both the univariable and multivariable models were very unstable. Thus, we do not feel confident that the statistical modeling has provided either a satisfactory explanation for the study-to-study variability in results or credible estimates of adjusted relative risk.

One group of patients fared consistently better in Canada than in the U.S., those with end-stage renal disease.4,28-33 Whether in hemodialysis programs, peritoneal dialysis, or after receipt of renal transplants, Canadians survive longer. The larger proportion of Americans than Canadians who begin dialysis treatment confounds interpretation of this finding. Perhaps Americans fare worse because a larger number of sicker patients enter dialysis. On the other hand, it may be that the larger proportion of Americans on dialysis reflects a lower threshold to start dialysis, and thus a less sick dialysis population. The limited available evidence suggests that thresholds for dialysis are in fact similar in the two countries.58 Furthermore, two high-quality studies that included extensive adjustment for comorbidity29,33 still show substantially lower mortality in Canadian patients, suggesting that imbalance in risk cannot explain superior Canadian outcomes.

Nevertheless, the weight of the evidence strongly suggests that Canadian end-stage renal patients truly have higher survival than those in the US. The explanation for this difference may lie in differences in the ownership of dialysis facilities. Virtually all Canadian dialysis care is not-for-profit, while for-profit providers deliver approximately 75% of American care for end-stage renal failure. A systematic review has shown a higher mortality in patients undergoing dialysis in for-profit centres.59

Despite the limitations of the available studies, some robust conclusions are possible from our systematic review. These results are incompatible with the hypothesis that American patients receive consistently better care than Canadians. Americans are not, therefore, getting value for money; the 89% higher per-capita expenditures on health care in the United States does not buy superior outcomes for the sick.

Canadian health care has many well-publicized limitations. Nevertheless, it produces health benefits similar, or perhaps superior, to those of the US health system, but at a much lower cost. Canada’s single-payer system for physician and hospital care yields large administrative efficiencies in comparison with the American multi-payer model.60 Not-for-profit hospital funding results in appreciably lower payments to third-party payers in comparison to for-profit hospitals61 while achieving lower mortality rates.62 Policy debates and decisions regarding the direction of health care in both Canada and the United States should consider the results of our systematic review: Canada’s single-payer system, which relies on not-for-profit delivery, achieves health outcomes that are at least equal to those in the United States at two-thirds the cost.