Sentinel Q&A:
How does external validity come into play when planning and conducting projects in the database? A typical Sentinel analysis includes data from multiple large national and regional commercial health plans, a state Medicaid plan, and Medicare fee-for-service plan. Given the demographic and geographic diversity of the source data and the focus on internal validity, external validity is typically not an explicit consideration when planning or conducting Sentinel projects. That said, attention is paid to the relevance of the study population when finalizing the analysis plan and deciding on eligibility criteria (e.g. high risk individuals, true new users of the drug). These considerations also shape whether specific data partners are included in the analyses. Finally, if outlier results are observed from some data partners, the operations center may examine how the outlier populations differ from the other study populations.
What target population, if any, underlies most analyses? For the most part, the FDA is focused on studying treatment effects on individuals “treated” with drugs in the United States. The major gap with respect to this target is the lack of data on the uninsured and individuals with Medicaid, but there are ongoing efforts to fill this gap, such as the inclusion of more Medicaid data.
Are there ways to generalize the findings of the nodes to the network? Depending on the specific analysis, the operations center can always combine the outcome and treatment data across the sites to obtain a “network-wide” effect estimate. It is also possible to include a specific portion of the network in an analysis, e.g., running the analysis only in the Medicare fee-for-service data when examining the treatment effects in the elderly.
How easily can node-specific estimates be transported between nodes or to external populations? While it is theoretically possible to standardize node-specific estimates to one population (e.g., the entire US population), it has not yet been implemented within Sentinel. However, Sentinel routinely performs pre-specified subgroup analyses to identify treatment effect heterogeneity.
Are choices ever made to maximize target validity, rather than internal validity or precision? Given that all the data partners are from the US, and the general focus is on general US population treated with the medications, requesters generally prioritize internal validity and precision..