Evaluating the appropriateness of electronic information resources for learning in pre-clerkship medical education : an eye tracking study
Metadata[+] Show full item record
Current U.S. medical students have begun to rely on electronic information repositories -- such as UpToDate, Access Medicine, and Wikipedia -- for their pre-clerkship medical education. However, it is unclear whether these resources are appropriate for this level of learning due to factors involving information quality, level of evidence, and the requisite knowledge base. This study evaluated the appropriateness of electronic information resources from a novel perspective: the amount of mental effort learners invest in interactions with these resources and the effects of the experienced mental effort on learning. Eighteen first-year medical students read about three unstudied diseases in the three above-mentioned resources (a total of 54 observations). Their eye movement characteristics (i.e., fixation duration, fixation count, visit duration, and task-evoked pupillary response) were recorded and used as psychophysiological indicators of the experienced mental effort. Post reading, students' learning was assessed with a multiple-choice test. Eye metrics and test results constituted quantitative data that were analyzed according to the repeated Latin square design. Students' perceptions and observations of their interactions with the information resources constituted qualitative data that were also obtained. Participants' feedback from semi-structured interviews and recordings of students' information acquisition behaviors were reviewed, transcribed, and open coded for the emergent themes. Compared to Access Medicine and Wikipedia, UpToDate was associated with significantly higher values of eye metrics suggesting higher mental effort experienced by learners when using this resource. No statistically significant difference between the amount of mental effort and learning outcomes was found. More so, descriptive statistical analysis of the knowledge test scores suggested similar level of learning regardless of the information resource used. Students' feedback and observations of their behaviors were informative in understanding and interpreting the differences in quantitative findings. Judging by the learning outcomes, all three information resources were found appropriate for learning. UpToDate, however, when used alone, may be less appropriate for first-year medical students' learning as it does not fully address their information needs and is more demanding in terms of invested cognitive resources.