A Trustworthy Explainable Transformer-Based Ensemble Learning Framework For Mental Health Treatment Prediction

Abstract

Although the worldwide burden of mental illness is increasing, a significant percentage of people are not seeking professional help; factors differentiating those who seek help from those who do are not easily modelled using classical statistical tools. In this study, a transformer-based model for predicting mental health treatment-seeking behavior from workplace and demographic survey data, using an explainable approach, is presented. We propose a modified version of TabTransformer, a transformer-based architecture, incorporating a dedicated classification (CLS) token, learned column-identity embeddings, class-weighted loss, and label smoothing for modeling the contextual dependency among the categorical items in the survey with a large-scale dataset of 292,364 responses over fifteen categorical features (demographic, occupational, and attitudinal). The model is compared with five classical and ensemble baselines (Logistic Regression, Random Forest, XGBoost, LightGBM, CatBoost) with the same train/validation/test protocol, and the individual architectural modification of the model is determined by a systematic ablation study. Tab Transformer achieves the highest overall accuracy (0.785), macro-F1 (0.784), and macro-AUC (0.877) of all the models evaluated, just slightly better than CatBoost and other gradient-boosting models, but significantly better than Logistic Regression. We use SHAP and LIME to provide global, per-class, and instance-level explanations of the model to highlight the survey items that most strongly determine treatment-seeking results, particularly in cases where deep tabular models are opaque. The findings show that the TabTransformer, a transformer-based model able to perform as well as thebaseline study, also provides a built-in way to interpretability, making it suitable as a starting point for building explainable decision support tools in mental health screening scenarios.