HC Observational Medical Outcomes Partnership (OMOP)

The Observational Medical Outcomes Partnership (OMOP) is a public-private partnership, chaired by the FDA, managed by the Foundation for NIH, and supported by a concortium of pharmaceutical companies, with the mission to conduct methodological research on the appropriate use of observational healthcare data for studying the effects of medical products. OMOP has completed a series of experiments to evaluate the performance of various statistical algorithms for estimating the average treatment effect in real-world observational data, including administrative claims and electronic health records, as well as through simulation models. Specifically, the OMOP research team executed 7 different observational analysis designs (including cohort, case-control, and self-controlled case series, each with hundreds of analysis design variants) against 5 real observational databases and 16 simulated datsets for a ground truth set of 399 drug-outcome pairs (165 positive controls and 234 negative controls). Each analysis (a method applied to a database for a drug-outcome pair) produced a effect estimate (eg. relative risk) and associated standard error. Initial evaluation of these results have demonstrated promise in identifying methods that out-perform alternatives, differences between databases, and examples of drug-outcome pairs that are regularly misclassified. There is widespread belief that the OMOP team has only scratched the surface of what can be learned from these experimental results. In keeping with OMOP's public mission, the analysis algorithms, simulated data, and all summary results of these experiments have been made publicly available for the broader research community to explore and study further. One potential opportunity for the SAMSI working group is to leverage this publicly available dataset to collaborate on developing methods evaluation strategies and to assess performance on which methods work best on which data for which type of scenarios of drug exposure and outcomes. Establishing alternative methodological frameworks for evaluating performance of observational analyses and demonstrating how these frameworks are applied to empirical data could benefit researchers in statistics, epidemiology, and informatics through peer-reviewed publications and scientific presentations, and would also directly serve the public health need of learning how to generate reliable evidence from observational data to study the effects of medical products.