Treatment fidelity of technology-enhanced reading therapy (CommuniCATE) for people with aphasia

Background Treatment fidelity (TF), that is, the degree to which the treatment delivery has adhered to protocol, is an important aspect of establishing treatment validity and reliability. Research has shown that establishing TF is only done in a small percentage of aphasia treatment studies. Aims This project supports the work of the CommuniCATE study, which explored the benefits of technology-enhanced aphasia therapy on participants’ reading, writing, speech and conversation skills. It examines the TF of the Reading strand of the CommuniCATE project by assessing whether the therapy adhered to the protocol. The following research questions were asked: Does treatment delivery adhere to treatment protocol? Does the degree of TF vary according to the person delivering the therapy (i.e. student therapist or qualified therapist)? Does the degree of TF vary over time (early treatment sessions compared with later treatment sessions)? Was the checklist tool reliable? Methods & Procedures This study assessed the fidelity of 38 retrospective video recordings of therapy. It used a checklist measure of criteria to which the delivery of the sessions should adhere, and against which the sessions were rated. Participants were the people with aphasia receiving therapy, the students and qualified speech and language therapists delivering therapy, and the independent raters assessing the sessions. A sample of sessions was randomly chosen, including sessions delivered by qualified therapists and by students, and sessions from different time points in the treatment process. The fidelity was rated by the first author, and the fidelity rating calculated as a percentage. Comparisons in fidelity scores for the different variables were drawn using Mann–Whitney tests. The reliability of the checklist was assessed through inter and intra-rater reliability testing, and the results were analysed using Kappa statistics. Outcomes & Results High fidelity was found across all therapy conditions with a mean score of 98.2%. Fidelity scores were not affected by the administrator of therapy; sessions delivered by qualified and student therapists were rated equally highly. There was a small but significant effect of time, with later treatment sessions scoring more highly than earlier sessions. However, scores across both periods > 90%. Inter-rater reliability found a high percentage agreement of 93.3% and a Poor Kappa agreement level. Intra-rater agreement found a high percentage agreement of 97.3% and a Fair Kappa agreement level. Conclusions & Implications The CommuniCATE reading therapy was implemented as per the protocol across time points, and withstood delegation to students. The high fidelity and good reliability scores have positive implications for the study's validity and reliability, and for the study's replication.