Intelligent Tutor System (ITS) for Children with Autism using Generative AI

Primary Supervisor: Dr Charaka J Palansuriya

Additional Supervisor: Dr Aurora Constantin (School of Informatics)

A high percent of the children with ASD have co-morbidities, including learning disability. Research revealed that children with ASD can benefit from the educational interventions. Particularly, technology-based interventions are very promising as children with ASD have affinity toward technology. In addition, technology has a series of advantages for these children. For example, technology acts an interface between individuals with ASD and other people and that creates emotional and social distancing which is likely to diminish the anxiety. Another advantage is that technology is well-placed to customise the interventions to the child’s particular needs and interests. Moreover, the autistic population is known as being extremely heterogenous. Thus, children with ASD have highly varying learning abilities. While children with Asperger Syndrome (one of the conditions included in ASD) have normal language and above average intelligence, other autistic children are completely nonverbal and present severe intellectual disability. Consequently, these children require personalised one-to-one guidance to learn both life-long skills and academics skills. Training adequate number of teachers and assistants to be specialised in providing this type of guidance is challenging to say the least.

While technology could provide support to children with ASD, given the heterogeneity of the population, it is unlikely that a software that works a predefined way could work well for all these children.

Currently, there is on going research at EPCC to use children’s Facial Emotion Recognition (FER) to identify when and how to, in real time, dynamically adjust the support provided to children with ASD when using a software based tutoring system [1]. With recent advancement in Generative AI technologies, personalised academic, social and communication related educational content could be dynamically generated in real time to support both children with ASD as well as Typically Developing (TD) children. Children themselves could participate in co-creating such content – for example, indicate what they like to learn and see in terms of text, images, animations and videos. This research would need to focus on fine tuning existing pre-trained generative AI models for generating content suitable for children. Also, this means that AI safety would be a key focus in this research.

Overview of research area

Generative AI technologies have made significant advancements recently that it is possible to generate human curated level content in terms of text, images, animations and even videos. ChatGPT and DALLE-2 are good examples of such generative AI technologies for creating new text and images respectively. However, both ChatGPT and DALLE-2 are closed source and mainly provide paid API level access when integrating into software applications. Restricting content for generating safe and suitable material for children would be based on what (if any) related API features available. Also, the input content will have to be submitted to a Cloud infrastructure operated by a commercial third-party entity (e.g., OpenAI/Microsoft). It is also possible to experience periods of service outage or high latency when generating content.

What this research will investigate is the use of open-source generative AI models and related technologies to generate the content for an Intelligent Tutoring System (ITS) for Children with ASD. Stable Diffusion [2] provides an open-source technology similar to DALLE-2 for image generation based on text input. Open Assistant [3] is a similar generative AI technology to ChatGPT for generating human curated level text content. Also, an open-source version of GPT3 called GPT-J [4] is now available for generative AI based textual content creation. There likely be many more open-source efforts like these emerging in next few months. What this research will aim to do is to adapt these open-source generative AI technologies and their corresponding pre-trained models in a child-safe and coherent way to create content suitable for children with ASD and typically developing children. It is likely that some fine tuning of the models will be necessary with children specific datasets – for example, by using existing free stories available for children. In addition to the AI safety aspect, a research objective would be to create AI models that can be inferenced locally within a device itself – for example, within a laptop/tablet with powerful GPUs (e.g., laptop with NVIDIA GPUs or Apple silicon M1 or above macbooks/iPads). This means all data (including sensitive ones) stays locally within the device and does not need to be submitted to third party entities.

Potential research questions

• Could the open-source generative AI technologies and models be adapted to create content suitable for educational needs of children with ASD as well as typically developing children?
• What are the techniques required to implement required AI safety aspects when creating content for children?
• How could FER techniques be integrated to create suitable content in real time?
• What are the appropriate ways to perform and evaluate usability of the learning technologies developed?

Student Requirements:

A UK 2:1 honours degree, or its international equivalent, in a relevant subject such as computer science and informatics, physics, mathematics, engineering, biology, chemistry and geosciences.

You must be a competent programmer in at least one of C, C++, Python, Fortran, or Java and should be familiar with mathematical concepts such as algebra, linear algebra and probability and statistics.

Standard University English language requirements.

Student Recommended/Desirable Skills and Experience

• A good understanding in use of machine learning techniques – particularly the Transformer model (see [5]) and its derivatives
• Understanding and experience in Human Computer Interaction (HCI) – particularly Child Computer Interaction (CCI)
• Knowledge and experience in applying child psychology
• Experience in performing usability studies, particularly with children

References

1. Real-time Feedback based on Emotion Recognition for Improving Children’s Metacognitive Monitoring Skill, https://dl.acm.org/doi/10.1145/3501712.3538831
2. Stable Diffusion, https://github.com/CompVis/stable-diffusion
3. Open Assistant, https://open-assistant.io/
4. GPT-J, https://huggingface.co/EleutherAI/gpt-j-6B
5. Attention Is All You Need, https://arxiv.org/pdf/1706.03762.pdf