Researchers from DTU have found a new way to examine data from eye trackers.
“Our data-driven method is based on machine learning and can contribute to the diagnosis of patients and to determining whether a treatment or – further down track – newly developed medicine has the desired effect,” says Paolo Masulli, a former postdoc at DTU Compute and now employed by iMotions, who has been involved in developing software for data analysis and modeling of the biometric data.
The research result is a component of a recently completed project supported by the Innovation Fund Denmark, and it was published in the international neuroscience journal Cortex.
When patients are examined for neurological or psychiatric disorders using eye-tracking, researchers present the patients with a series of photos or videos with faces on a screen. The eye-tracking sensor then tracks the patient’s eye movements and keeps track of specific areas in the images. The results can be seen on a heatmap; The number of times patients spend looking at certain areas of the image will affect the color or the heatmap.
“Researchers typically define and frame which areas are of interest. However, there is no exact science that defines the size and location of these fields in the images. There is a lot of subjectivity in it. Our method is different. We do not define specific areas in advance, but let the data speak and our method thereby provides us with a more objective assessment of the patient’s eye movements,” says Paolo Masulli.
Patients aged 18 to 25– from the data of 111 outpatient psychiatric patients, which the Swedish university partner, Gillberg Neuropsychiatry Center has recorded– showed symptoms of autism, depression, or ADHD and wanted to participate in the research project and make their anonymized data available for research.
In the trial, patients answered some standard clinical tests, which place them on numerical scales according to the severity of their symptoms. They were then presented with a series of black-and-white images in which the person in the image expresses joy, anger, or looks neutral, while the eye-tracker collected data from the entire image.
Subsequently, all heatmaps were analyzed using machine learning, where the most important components (points on the face) were statistically identified from the data set.
The researchers then used the components and numbers from the clinical tests for the symptoms to set up a mathematical model that links the components to the degree of autism, depression, and ADHD. Based on the model, new data-driven heatmaps have been created that correspond to a specific level of the clinical tests.
“Our method confirms the trends that recent studies have found and shows that machine learning can be used in neuroscience research,” says Paolo’s supervisor, Associate Professor at DTU Compute Tobias Andersen.
This method will be able to be used in clinics, where doctors often meet patients who have not already been given a diagnosis. Implementing the algorithm in eye-tracking software will enable it to show how the patient ranks on the scales of autism, depression, and ADHD.
Often there is also a question of comorbidity, where a patient has more than one psychiatric or
Tobias Andersen says, “Some people with autism want to get better at social interactions, and here the method can be used to detect eye movements and see how the patient is affected by his or her autism and give him or her automatic feedback. And, if you train patients to have an easier time looking at faces, you will be able to use the method to see if the training works.”
The research result contributes significant new knowledge, according to the Swedish partner, Gillberg Neuropsychiatry Center at the University of Gothenburg:
“Most of the research done using eye-tracking has used arbitrary, experiment-defined areas of interest. Our approach constitutes an important advance, because it is objective, data-driven, and it enables the application of a dimensional (multilateral) approach to the study of neuropsychiatric conditions and their correlates in terms of gaze behavior,” says Nouchine Hadjikhani, Professor at the University of Gothenburg and Harvard Medical School.
Gillberg Neuropsychiatry Centre points out this new approach also can generate innovative hypotheses:
“For example, we observed that a reduction of a left visual field bias is associated with depression. Whether this is this a state, or a trait, and whether this could be used as a biomarker for the follow up of treatment effects related to depression is one of the questions that could, for instance, be explored in future studies,” says Nouchine Hadjikhani,