Predicting engagement and churn in digital health for micro-entrepreneurs : limits of log-only data analysis

Abstract

Micro-entrepreneurs, due to demanding schedules and high stress levels, present a unique and high-attrition demographic for digital health interventions (DHIs). While these platforms generate extensive user log data, robustly predicting engagement and churn remains complex due to the influence of unobserved external factors. This study addresses this gap by applying machine learning (ML) techniques and a dual methodology - descriptive analysis (SRQ1) and predictive modelling (SRQ2) - to analyse eight weeks of log-derived static and dynamic metrics from a DHI targeting micro-entrepreneurs. Findings demonstrate that ML models trained exclusively on log-derived features, irrespective of the algorithm, achieved low F1-scores and AUC values only marginally better than random chance. This outcome empirically validates the study’s core hypothesis, revealing the insufficiency of log-only data for robust behavioural prediction in high-context populations. This limitation is directly attributed to the absence of critical contextual and psychological variables, such as external stressors and motivational factors. The study provides evidence for the limitations of the log-only paradigm and argues for a strategic shift toward multi-modal data integration, combining objective usage logs with subjective and contextual inputs. Design recommendations focus on optimizing engagement by prioritizing the perceived relevance of micro-tasks and implementing personalized, context-aware guidance. These insights inform the development of more effective, sustainable, and context-aware digital health interventions for time-constrained user cohorts.