Valeriano, Maria Gabriela and Matran-Fernandez, Ana and Veiga Kiffer, Carlos Roberto and Lorena, Ana Carolina (2024) Understanding the performance of machine learning models from data-to patient-level. Journal of Data and Information Quality. DOI https://doi.org/10.1145/3687267
Valeriano, Maria Gabriela and Matran-Fernandez, Ana and Veiga Kiffer, Carlos Roberto and Lorena, Ana Carolina (2024) Understanding the performance of machine learning models from data-to patient-level. Journal of Data and Information Quality. DOI https://doi.org/10.1145/3687267
Valeriano, Maria Gabriela and Matran-Fernandez, Ana and Veiga Kiffer, Carlos Roberto and Lorena, Ana Carolina (2024) Understanding the performance of machine learning models from data-to patient-level. Journal of Data and Information Quality. DOI https://doi.org/10.1145/3687267
Abstract
Machine Learning (ML) models have the potential to support decision-making in healthcare by grasping complex patterns within data. However, decisions in this domain are sensitive and require active involvement of domain specialists with deep knowledge of the data. In order to address this task, clinicians need to understand how predictions are generated so they can provide feedback for model refinement. There is usually a gap in the communication between data scientists and domain specialists that needs to be addressed. Specifically, many ML studies are only concerned with presenting average accuracies over an entire dataset, losing valuable insights that can be obtained at a more fine-grained patient-level analysis of classification performance. In this paper, we present a case study aimed at explaining the factors that contribute to specific predictions for individual patients. Our approach takes a data-centric perspective, focusing on the structure of the data and its correlation with ML model performance. We utilize the concept of Instance Hardness, which measures the level of difficulty an instance poses in being correctly classified. By selecting the hardest and easiest to classify instances, we analyze and contrast the distributions of specific input features and extract meta-features to describe each instance. Furthermore, we individually examine certain instances, offering valuable insights into why they offer challenges for classification, enabling a better understanding of both the successes and failures of the ML models. This opens up the possibility for discussions between data scientists and domain specialists, supporting collaborative decision-making.
Item Type: | Article |
---|---|
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 25 Sep 2024 10:39 |
Last Modified: | 25 Sep 2024 10:40 |
URI: | http://repository.essex.ac.uk/id/eprint/38492 |
Available files
Filename: ACCEPTED-PDF53369264-1114352549.pdf