Evidence-Driven Simulated Data in Reinforcement Learning Training for Personalized mHealth Interventions

Physical inactivity is a major preventable cause of non-communicable disease and premature mortality. Mobile health interventions can promote physical activity, but their effectiveness depends on the ability to adapt to user’s context and motivation. Reinforcement learning (RL), particularly contextual bandits (CBs), offers a promising framework for such adaptive personalization. However, in practice, RL-based models face the cold start problem (CSP), due to the lack of initial training data. This study examines whether theory-driven simulated data can mitigate the CSP in training RL systems for personalized physical activity recommendations. A scoping review of 18 empirical studies on the Integrated Behavioral Change Model (IBC) provided population parameters for key constructs, used to simulate 2000 virtual users via multivariate modeling and structural equation calibration. A CB algorithm with an ε-greedy policy was trained with this dataset and compared with data from real world pilot using the Apptivate mHealth web-app (n = 588). Results showed close alignment between simulated and real behaviors. Our findings demonstrate that behaviorally informed synthetic data can effectively be used to train RL algorithms, offering an interpretable, sustainable, scalable, and privacy-safe solution to the CSP in personalized digital health interventions.