Hsor 2011-3.pub

The evidence behind your decisions
Health Services & Outcomes Research, Issue 2011/3
COMPARING APPLES AND ORANGES: THE USE OF PROPENSITY SCORES IN OBSERVATIONAL STUDIES INSIDE THIS ISSUE
T wo heart surgeons walk into a room. The first surgeon says: "I just finished my 100th heart surgery!" The second surgeon replies: "Oh yeah, I finished my 100th heart surgery last week. Only 10 of my pa- tients died within 3 months of surgery." First surgeon smugly responds: "Only 1. Observational studies & selection 5 of mine died, so I must be the better surgeon." Second surgeon says: "My patients were probably older and had a higher risk than your patients." 2. What is propensity score? Such non-randomised comparisons give rise to apples-and-oranges scepticism. Observational Studies and Selection Bias
Traditional methods to adjust for selection bias
Matching: We can match treatment and control group pa-
Except by chance, patients in different comparator groups tients based on selected characteristics using a case control tend to differ in non-randomised studies. Although randomi- design. However, this becomes impractical when there are a sation can produce relatively comparable treatment groups, large number of covariates. observational studies are used because: Stratification: We can stratify the treatment and control
♦ Randomised controlled trials with strict inclusion criteria groups according to selected characteristics. However, the may have limited external validity number of sub-categories will increase exponentially, leading ♦ Data are widely (increasingly) available and can reduce to a problem of having few subjects in each cell to have a cost and time to get answer meaningful comparison. ♦ They enable examination of real life situations Multivariate Adjustment: We can statistically adjust for
♦ Large sizes permit investigation of exposures with smaller baseline differences between the groups simultaneously. However, this does not resolve the issue that the people get- ting treatment may be systematically different from the con- Observational studies are common in health services research trol group in ways that affect outcome. but as treatment assignment is outside the control of the investigator, the potential for biases is higher. What is propensity score (PS)?
Propensity score (PS) helps to balance the data sets by trans-forming an apples-to-oranges outcomes comparison into an apples-to apples comparison. For each patient, we can esti-mate the propensity toward (0≤ PS ≤ 1) belonging to the treatment group versus non-treatment. To derive the propensity scores, we first construct a logistic regression model that represents the treatment allocation decision. Therefore, we should consider including any vari-ables that have a relationship to the treatment decision. After which, we need to check whether the scores for the compara-tor groups overlap reasonably (Fig. 1). If they do, we can proceed to use the scores in the following ways. If not, it just means the two groups are too different to offer any sensible comparison. Ways to apply propensity scores
We can use the PS in our analysis of treatment effects through matching, stratifying and inclusion of the score or the strata in a regression equation. Figure 1: Non-overlap of the propensity score distributions among exposed and unexposed subjects* Matching: We can match the score for the first treatment
patient to all control patients within a given caliper around
* In this example subjects with low propensity scores are the score. Matched patients may not be exactly similar but never exposed while subjects with high propensity scores are overall probability of being treated is close. always exposed Stratification: PS can be stratified. The number of strata will
Ref: Stürmer T et al. A review of the application of propensity depend on how many participants are available. Subsequent score methods yielded increasing use, advantages in specific analyses can be conducted within each strata. settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epide- Regression adjustment: The continuous PS or quintile can
miol. 2006 May;59(5):437-47 be included as a dependent variable in the final model for testing the effectiveness of treatment. "Does aspirin use convey a survival benefit in patients with known or suspected coronary disease?"
To answer this question, a prospective, observational cohort study was conducted. Because aspirin use was not ran-domly assigned, potential confounding and selection biases were accounted for by developing a PS for aspirin use. A logistic regression equation with 34 covariates was used to predict the PS of aspirin use. Each aspirin user was matched to a non–user with the closest PS. Table 1 illustrates that before PS matching, the baseline characteristics were dissimilar. The profile of aspirin users indicated higher mortality risk. However, post-PS matching, we see an apple-to-apple comparison with differences becoming statisti-cally insignificant. Table 1: Selected Baseline Characteristics Before & After PS Matching Before Matching (%)
After Matching (%)
No Aspirin
No Aspirin
Variable
(n=2310)
(n=3864)
(n=1351)
(n=1351)
Prior coronary artery disease Congestive heart failure Beta-blocker usage Ace Inhibitor usage In a simple unadjusted comparison, there was no association between aspirin use and mortality (4.5% versus 4.5%). How-ever, in the multivariate analysis, aspirin use was associated with reduced mortality. In further analysis using matching by propensity score, 1351 patients who were taking aspirin were found to be at lower risk for death than 1351 patients not us-ing aspirin (4% versus 8%). Ref: Gum PA et al. Aspirin use and all-cause mortality among patients being evaluated for known or suspected coronary ar-
tery disease. JAMA. 2001 Sep 12;286(10):1187-94.
Final word
Woan Shin, Principal Research Analyst, has been a principal investigator on several PS cannot replace properly designed, ethical randomised tri- projects. She has conducted technical als. However, when randomisation is not practical, by explic- analyses for studies evaluating quality im- itly modelling the treatment allocation process, it helps the provement interventions, chronic disease analyst ensure an apple to apple comparison. interventions, health resource utilization and costs. Her current research interests include Limitations
evaluating the impact of risk adjustment methods on outcomes and the application However, relatively large sample sizes are required to facili- of economic evaluations in healthcare. She tate the use of PS in stratified or matched case analyses. was one of three recipients of the ISPOR Furthermore, when important variables influencing selection Contributed Research Award for Best New are not collected, the PS may not reflect the selection proc- Investigator Podium Presentation in 2007. Prior to joining NHG, ess, and will be seriously degraded. Hence, to adjust for un- Woan Shin worked as a Research Economist with the Ministry of observed covariates, we may have to look to other econo- National Development metric methods such as Instrumental variables (IV) or Differ-ences-in-Differences (DID). Tan Wan Shin, BSocSc (Hons) (Economics), MSocSc (Economics)
Our past newsletters are available under "Publications" Feedback and enquiries: [email protected] section at http://www.hsor.nhg.com.sg/ 8 June 2011

Source: https://hsor.nhg.com.sg/publications/newsletters/NewslettersDocLibrary/issue-19.pdf