Improving unified information extraction in Chinese mental health domain with instruction-tuned LLMs and type-verification component

Background: Extracting psychological counseling help-seeker information from unstructured text is crucial for providing effective mental health support. This task involves identifying personal emotions, psychological states, and underlying psychological issues but faces significant challenges. These challenges include the sensitivity of mental health data, the lack of Chinese instruction datasets, and the difficulties large language models (LLMs) encounter with complex natural language understanding tasks. Objective: This study aims to address these challenges by developing a unified information extraction framework for Chinese mental health texts. Specifically, it leverages instruction-tuned LLMs and incorporates a novel type-verification (TV) component to improve performance while minimizing computational demands. Methods: We first constructed a Chinese mental health domain instruction dataset for mental health information extraction using synthetic data generated by ChatGPT, guided by psychology experts. This dataset includes self-reported statements from psychological counseling help-seekers, capturing their personal situations, emotions, thoughts, and experiences. Subsequently, we fine-tuned open-source LLMs on this dataset to perform named entity recognition, relation extraction, and event extraction. To address errors and omissions in the extracted information, we introduced a type-verification component. This component employs a lightweight model with significantly fewer parameters to verify the extracted types. The verification results were then fed back into LLMs for further refinement. Results: Experimental results demonstrate that our framework achieves outstanding performance in mental health information extraction. The type-verification component significantly enhances extraction accuracy while reducing computational resource requirements through the use of a lightweight model. By combining robust instruction-tuned LLMs with an efficient type-verification component, our approach delivers exceptional results. Conclusion: This study presents a novel and efficient framework for tackling the challenges of mental health information extraction in Chinese texts. By integrating instruction-tuned LLMs with a lightweight type-verification component, our approach significantly improves extraction accuracy and computational efficiency. This framework holds promise for supporting scalable, automated mental health support systems, advancing both research and practical applications in the mental health domain.