Predicting missed health care visits during the COVID-19 pandemic using machine learning methods: evidence from 55,500 individuals from 28 European countries

Abstract

Abstract

Background Pandemics such as the COVID-19 pandemic and other severe health care disruptions endanger individuals to miss essential care. Machine learning models that predict which patients are at greatest risk of missing care visits can help health administrators prioritize retentions efforts towards patients with the most need. Such approaches may be especially useful for efficiently targeting interventions for health systems overburdened during states of emergency.

Methods We use data on missed health care visits from over 55,500 respondents of the Survey of Health, Ageing and Retirement in Europe (SHARE) COVID-19 surveys (June – August 2020 and June – August 2021) with longitudinal data from waves 1–8 (April 2004 – March 2020). We compare the performance of four machine learning algorithms (stepwise selection, lasso, random forest, and neural networks) to predict missed health care visits during the first COVID-19 survey based on common patient characteristics available to most health care providers. We test the prediction accuracy, sensitivity, and specificity of the selected models for the first COVID-19 survey by employing 5-fold cross-validation, and test the out-of-sample performance of the models by applying them to the data from the second COVID-19 survey.

Results Within our sample, 15.5% of the respondents reported any missed essential health care visit due to the COVID-19 pandemic. All four machine learning methods perform similarly in their predictive power. All models have an area under the curve (AUC) of around 0.61, outperforming random prediction. This performance is sustained for data from the second COVID-19 wave one year later, with an AUC of 0.59 for men and 0.61 for women. When classifying all men (women) with a predicted risk of 0.135 (0.170) or higher as being at risk of missing care, the neural network model correctly identifies 59% (58%) of the individuals with missed care visits, and 57% (58%) of the individuals without missed care visits. As the sensitivity and specificity of the models are strongly related to the risk threshold used to classify individuals, the models can be calibrated depending on users’ resource constraints and targeting approach.

Conclusions Pandemics such as COVID-19 require rapid and efficient responses to reduce disruptions in health care. Based on characteristics available to health administrators or insurance providers, simple machine learning algorithms can be used to efficiently target efforts to reduce missed essential care.