Assessing Fidelity of Implementation to a Technology-Mediated Early Intervention Using Process Data
| Title: | Assessing Fidelity of Implementation to a Technology-Mediated Early Intervention Using Process Data |
|---|---|
| Language: | English |
| Authors: | Helsabeck, Nathan P. (ORCID |
| Source: | Journal of Computer Assisted Learning. Apr 2022 38(2):409-421. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 13 |
| Publication Date: | 2022 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305A150274 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Early Childhood Education Preschool Education |
| Descriptors: | Fidelity, Program Implementation, Early Intervention, Handheld Devices, Preschool Children, Preschool Curriculum, Supplementary Education, Reading Instruction, Data Use, Preschool Teachers, Web Based Instruction, Educational Technology |
| DOI: | 10.1111/jcal.12621 |
| ISSN: | 0266-4909 |
| Abstract: | Background: Process data, data generated by a user's interaction with a web-based application, is an emerging tool in educational research. The current study explores using process data as a measure of implementation fidelity to a randomized control trial (RCT) of the Read It Again Mobile (RIA-M) curricular supplement. Objectives: To determine the extent to which teachers implemented RIA-M and to assess the utility of using process data in the assessment of fidelity. Methods: The RCT involved 30 pre-kindergarten classrooms with a sample of n = 216 students. RIA-M provides a curricular supplement which teachers may incorporate into classroom instruction and is delivered via a tablet computer. Pre and post literacy assessments are used to determine treatment effect. Process data, produced from teacher interactions with the tablet, and classroom observations are used to assess fidelity. Results and Conclusions: Our findings indicate no difference between treatment and control students in the RCT. Yet, we find that process data provides unique fidelity information concerning treatment exposure, adherence, and quality of program delivery. Specifically, process data indicated that teachers did not demonstrate the same level of fidelity that was captured in classroom observations. This finding provides some evidence for the absence of an intervention effect. Major Takeaways: The current study improves our understanding of how web-based interventions may be assessed for implementation fidelity using process data. Further, process data offers a potentially reliable and scalable measure of fidelity for other web-based educational interventions. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2022 |
| Accession Number: | EJ1327603 |
| Database: | ERIC |
| ISSN: | 0266-4909 |
|---|---|
| DOI: | 10.1111/jcal.12621 |
| Published in: | Journal of Computer Assisted Learning |
| Language: | English |