New Continual Learning Methods Enhance Small Sample EHR Model Performance Using External Data
Leveraging External Data to Improve EHR-Based Predictions
Researchers have explored innovative approaches to enhance the accuracy of deep learning models trained on small electronic health record (EHR) datasets by incorporating information from external data sources. A recent study investigated the use of continual learning techniques to effectively borrow knowledge from analogous external datasets without directly merging sensitive medical records, which is often restricted by data use agreements.
Study Overview and Methods
The study focused on two clinical prediction tasks: identifying children likely to be diagnosed with autism spectrum disorder (ASD) by 18 months of age, and predicting 30-day rehospitalization risk among patients with end-stage renal disease (ESRD). Target datasets were obtained from Duke University’s EHR system, while external datasets included North Carolina Medicaid claims for the ASD task and the United States Renal Data System (USRDS) for the ESRD task.
To integrate external knowledge without directly combining datasets, the researchers applied two continual learning methods—elastic weight consolidation (EWC) and variational continual learning (VCL). Both methods use regularization techniques to incorporate parameters learned from external data into models trained on local target data, aiming to prevent the model from overfitting to patterns unique to the external source.
Key Findings
When training data from the target EHR was limited, models that borrowed parameters using EWC or VCL performed comparably to models trained exclusively on the full external datasets. In contrast, models trained solely on the small target datasets without borrowing external knowledge showed poorer predictive performance.
Furthermore, analyses revealed that borrowing models were better calibrated and produced predictions more consistent with those from external-data-only models, especially in scenarios with limited target data. This suggests continual learning approaches can help small-sample EHR models generalize more effectively by leveraging external information.
Implications for Clinical Machine Learning
These findings highlight a promising strategy for improving predictive model performance when access to large, local EHR datasets is constrained. By using continual learning methods, healthcare organizations may enhance model accuracy without violating data sharing restrictions or requiring costly data collection efforts.
Limitations and Considerations
While the results are encouraging, the study focused on two specific clinical tasks and datasets. Further research is needed to assess the generalizability of these methods across diverse medical conditions and data environments. Additionally, continual learning approaches must be carefully implemented to avoid incorporating biases or patterns that do not apply to the target population.
For more detailed information, see the original research report at PubMed.
Download the images in this article
