An intelligent tool created by a group of researchers from Kraków helps to diagnose autism in children. It has already been implemented by some European centers in Sweden and Great Britain
In their first game, the task of a child is to feed four cartoon characters. Whenever a character gets the food, it smiles widely. Those that are not fed are sad. You can also switch lamps on and off, and give windmill blades a turn.
In their second game, a child outlines simple shapes and fills a created area with colors.
After 15 minutes of play, we can tell, with a high degree of probability, whether the child develops normally or suffers from autism spectrum disorders.
The tool supporting early detection and fitted with artificial intelligence algorithms is being developed by a team of Harimata, a Kraków-based start up. It is led by Anna Anzulewicz, a psychologist from the Institute of Psychology, Warsaw University.
More subtle states
“I’m a psychologist with a flair for technology,” says Anna Anzulewicz. During her studies at Jagiellonian University she focused on cognitive psychology: memory, concentration, perception. Although professionally she mainly dealt with adults, privately she worked as a volunteer with autistic children.
In 2013 she was contacted by Paweł Jarmołkowicz, the founder of Harimata, and Mateusz Stec, a co-owner of Duckie Deck and a creator of games for children. They wanted to consult with her a project which consisted in detecting users’ emotions based on how they use a computer keyboard and mouse. It was about analyzing how fast and how smooth the movements of users’ hands were. For the next couple of months researchers, at random times of the day, asked users to assess their emotional state and then measured the parameters of their movements. It turned out that the applied methodology allowed to efficiently determine the level of agitation.
“But problems appeared when they started to look for customers. Only on-line casinos were interested,” says Azulewicz. “I think they wanted to know at what point their guests became nervous enough to stop playing so as to be able to bait them, to prevent them from leaving the game and to make them lose even more money.”
An average person is able estimate if a cup is heavy or light and if they should grab it firmly or take it gently to their hands. People diagnosed with autism have problems with that. That is why their movements are kind of “robotic”
But Harimata founders did not want to get involved in gambling. That was the reason why they reached out to Anzulewicz. They wanted her to help them out with identifying more subtle emotional states.
Harimata’s brainstorm
“The idea looked interesting. I hadn’t seen any solutions of the sort before. But I was aware that it would be a long-term project,” says Anna. “I was straightforward. I said it would be an uphill task requiring long research and that there was no chance for commercializing the results any time soon. Besides, I wasn’t interested in being commercially successful. What I cared about was a potential social value of the project. So me and the then Harimata team met and ran a brainstorm session.”
The idea came when they realized two things. On one hand, both therapists and parents admit that it is difficult to diagnose developmental disorders in children. But on the other hand, everybody knows that kids love playing with tablets. Why not to take advantage of that? They already had a technology that could collect information about emotions based on how we type on a keyboard and how we use a mouse. But they were also convinced that it was possible to get the same type of information from tablets.
Is it feasible to use commonly available tools, such as tablets or smartphones, to identify developmental disorder symptoms? Do children suffering from developmental disorders differ from children that develop normally as far as motor skill patterns are concerned?
What do you expect to find here?
Pilot studies followed. In the first round, the Harimata’s team examined almost a hundred autistic children aged two to five. Each of them played two customized games for 15 minutes. Then the artificial intelligence algorithm extracted various characteristics of their movements – including length, speed, smoothness, flowingness, number of swerves, precision – and analyzed the patterns. At the same time, it was also possible to discover how easily children get concentrated or distracted.
“Tablets proved to be great research tools,” says Anzulewicz. “It is extremely hard to do psychological testing with kids because they get bored pretty quickly. However, their interest in tablets allowed us to keep them in one place for 15 minutes and to collect accurate data with no extra effort.
Therapists were reluctant to their idea. “What do you expect to find here?” they said. “Autism has nothing to do with motion; it’s about problems with creating social relations!”
Looking for another benchmark
They were perfectly aware of that, but their intention was to find another benchmark. A benchmark that could be considered objective. Diagnoses are sometimes far-fetched, one of the reasons being the fact that you can obtain higher allowances for taking care of an autistic child. Sometimes children diagnosed with autism spectrum disorders placed in an integrated or special kindergarten develop normally.
What were the results?
“When we compare the heatmaps generated on tablet screens used by an autistic child and a child developing normally, we will notice that the movements of the former are much more angular and less precise,” describes Anzulewicz. “You can also notice that he or she got more distracted and concentrated on feeding only one cartoon character, without paying attention to the fact that others were sad. The efficiency of our algorithm ranged from 85 to 90 percent.
Conclusion: Children with autism spectrum disorders have different motion patterns than the kids who develop normally.
Kind of robotic movement
At that time, the Harimata team started collaboration with Jonathan Delafield-Butt from the University of Strathclyde, Glasgow, who also did research on motion patterns in autistic children. He believed that motor problems are one of the key symptoms of autism, although they are not described from a diagnostic perspective. They manifest in difficulties in planning and making movements.
“Reaching for a cup may seem a simple task, but it is impossible without extremely complicated computing processes happening in our brains,” explains Anzulewicz. “An average person is able estimate if a cup is heavy or light and if they should grab it tightly or take it gently to their hands. People diagnosed with autism have problems with that. That is why their movements are kind of “robotic”. Until recently, that fact was considered insignificant as far as autism was concerned. However, more and more studies seem to prove that motor disorders are primary for autism and that they are responsible for disrupting interaction with the environment. We learn through discovering capabilities of our body and properties of objects around us. To every action there is a reaction. If I don’t take a firm hold of a cup, I will drop it. I will then get feedback from the environment, which I will use next time to prevent the cup slipping away from my hand. But if the feedback is incompatible, if I plan something and yet fail to do it for an unknown reason, then I am not able to learn anything from that experience. Motor disorders are one of the reasons for which autistic children struggle with their emotional and social development.”
Girls’ camouflaging skills
Autism is a specific type of disorder. Some say it is just a different, unusual development model, similar to left-handedness. They claim that neither should be subject to a treatment. However, an early diagnosis and therapy may help a child a great deal, especially when it comes to creating social relations. A child must learn to live in a society and develop functioning strategies.
Some say autism is just a different, unusual development model, similar to left-handedness. They claim that neither should be subject to a treatment. However, an early diagnosis and therapy may help a child a great deal, especially when it comes to creating social relations.
“Screening tests have relied on questionnaires filled in by parents. Few people are able to properly evaluate their own children. Autism may be manifested in many different ways,” points out Anzulewicz. “Diagnosing is hard, especially in the case of girls. Although autistic girls represent only a quarter of all autistic children, they hide their problems really well, as they are trained for their social roles from their early life. It is also likely that the deficits resulting from autism are more easily masked with girls’ other capabilities, such as language skills. Be it as it may, girls are often diagnosed with autism only after they reach their school age and even after they become adults.
Stories against a backdrop of suffering
Many pediatricians do not know how to diagnose developmental disorders. |”How old is your son? Three? And he can’t talk? Don’t worry, boys start talking later than girls. Let’s wait for a bit longer,” Anna Anzulewicz have heard many such stories. That waiting results in unnecessary suffering of a children who know that something is wrong with them but can’t put their finger on it. Therapists are scarce. Visits need to be scheduled months in advance. This is why it is vital to identify early behavioral markers. To do that, the motion capture technology is sometimes used, but as this method is expensive, access to it is limited.
As it turned out, the same can be done with the use of a tablet. Clinical trials using the tool developed by the Harimata team are already conducted in Glasgow and Aberdeen, Great Britain as well as in Göteborg, Sweden, in one of the most renowned diagnostic and therapeutic centers for people suffering from autism. No results are available yet.
“The ultimate goal is to develop an application to identify disorders,” says Anzulewicz. “The tool is intended to provide support to specialists in their screening tests. Whenever a kindergarten teacher or a parent suspect an abnormal development of a child, they will be able to use it. They will know whether they should consult a specialist or not. The point is to have a child diagnosed and therapized by a specialist as soon as possible.
Money is not everything
None of the team members wants it to be a commercial solution. Researchers are financed by subsidies from the National Center for Research and Development and by European funds. Commercial companies expected immediate results. When they learned that it was impossible, they backed out.
“Developing a solution in the e-health area takes at least five years,” says Anzulewicz. “Investors, especially in Europe, don’t want to wait that long. We want to invest money coming mainly from subsidies and to devise an open source solution. We want our tool to serve the society and to be used for free.
Przeczytaj polską wersję tego tekstu TUTAJ