Project Description:
Nonspeaking autistic people (nonspeakers) can learn to communicate by typing with a single finger, offering an effective alternative to speech. However, they face significant challenges in controlling the precise finger movements required for typing. These motor challenges, along with attentional and sensory difficulties, mean that learning to type often takes years. While human therapists can help, this training is costly, not widely available, and can create dependency on specific individuals. As a result, few nonspeakers access the training needed to learn to type. This project addresses this gap by developing an innovative, affordable, automated system providing adaptive typing training that addresses the unique motor, attentional, and sensory challenges nonspeakers face. The collision of autism science and technology in this project unique, combining for the first time the lived experiences of nonspeakers with cutting-edge AI. Through a community-based participatory research framework, we will directly collaborate with nonspeakers who have learned to type, to create a highly personalized learning system — an approach not explored before.
The proposed solution is a tablet-based system that delivers engaging multimedia lessons from a freely accessible community-contributed library. Integrated with a large language model, the system will use the lesson context to generate questions whose difficulty is dynamically adjusted based on the user's history and real-time performance. Users will respond on a keyboard that employs AI to dynamically adjust the number and salience of keys available, guided by historical performance and real-time hand- and eye-tracking data. The system will also use this tracking information to provide audio-visual prompts to initiate and guide finger movements, offering attentional and regulatory support to maintain engagement. In Year 1, the system will be co-designed and tested with a small cohort of nonspeakers and practitioners. In Year 2, a longitudinal study will evaluate improvements in typing skills and communication.
This project is high risk because it attempts to create a complex, AI-driven system that addresses deeply individualized challenges in motor control and communication—a level of personalization that has not been achieved before. However, the potential rewards are substantial: accelerating the acquisition of typing and communication skills, thereby transforming the lives of millions of nonspeakers worldwide.
Target Audience:
Students/Trainees (long or intermediate trainees), Community Trainees / Short term trainees, Professionals and Para-Professionals, Family Members/Caregivers, Adults with Disabilities, Children/Adolescents with Disabilities/SHCN, Legislators/Policy Makers, General Public
