Cybathlon: An Innovative Guidance Device for the Visually Impaired
On October 27, 2024, during the Cybathlon finals, Team A-eye—led by pilot Salomé Nashed and ISIR researcher-instructors Ludovic Saint-Bauzel and Fabien Vérité—won a silver medal for their guidance device for blind and visually impaired individuals. This international competition shines a spotlight on technological innovations designed for people with disabilities, providing a real-world test for this ambitious project.
Congratulations to the team.
The Interview
This project, created in collaboration with Sorbonne University students, aims to make navigation for visually impaired individuals more intuitive through an innovative sensory approach. Here, Ludovic Saint-Bauzel and Fabien Vérité discuss the challenges, progress, and ambitions for this device.
What led you to design a kinesthetic feedback device for visually impaired people?
Ludovic Saint-Bauzel: As a researcher at ISIR, my initial studies focused on walking and standing assistance. This led me to explore kinesthetic communication—how users physically perceive a device and interact with it. Over the course of my research, I realized this concept could be applied to navigation for visually impaired individuals. In 2020, I developed a prototype, and the project gained new momentum when I learned about the Cybathlon in 2022.
Fabien Vérité: Since 2019, I’ve been an associate professor at ISIR, studying haptics, which examines how humans perceive their surroundings through touch. During my PhD, I focused on tactile feedback to improve balance. Currently, I am developing tactile feedback systems, particularly for surgical instruments.
Why did you choose to participate in the 2024 Cybathlon?
L. S.-B.: The Cybathlon is a competition where people with disabilities use technological devices to accomplish daily tasks. For us, this event offers a unique chance to test our technology in real-world conditions. It’s also an opportunity to showcase the technological advances that required intensive interdisciplinary collaboration within the university, involving robotics, computer science, and more. The Cybathlon allows us to highlight the contributions of our students, who played a significant role in developing the device.
F. V.: For me, participating in the Cybathlon is also a personal adventure. I attended the first edition in Zurich in 2016 when I was teaching high school. Seeing the incredible atmosphere and the assistive solutions for people with disabilities reminded me how research in this field can have a direct impact on people’s lives. That experience inspired me to return to research.
Can you explain how the technology you developed for visually impaired individuals works?
F. V.:Our device relies on kinesthetic communication, where information is conveyed through movement and physical effort, similar to guiding someone using gestures. The visually impaired user wears a harness equipped with a camera that maps the environment in real-time 3D. The system detects obstacles and calculates the optimal path. Then, a motor applies gentle pressure to the user’s hand to guide them in the right direction. For example, if the hand is nudged to the left, the user knows to turn that way. Once aligned, the pressure stops, allowing the user to move forward.
L. S.-B.: Unlike white canes or smart canes that signal obstacles to avoid, our device directly indicates where to go, making navigation more intuitive and less tiring. Our approach provides a more natural experience, similar to guidance from a guide dog or a person.
What are the advantages of kinesthetic feedback compared to other visual assistance technologies, such as audio or vibration-based methods?
L. S.-B.: Our method closely resembles the actual gesture of guiding a blind person. Kinesthetic feedback has the advantage of feeling much more natural, based on physical sensations we use daily, like guiding someone by the arm.
F. V.: It’s also less cognitively demanding. We’ve found that it makes navigation smoother and less tiring. The brain integrates this type of feedback more effectively because it aligns with a natural sensation of effort.
Salomé Nashed, pilote de l'équipe A-eye, en entraînement ©Sorbonne Université
What technical challenges did you encounter during the device’s development?
L. S.-B.: One of the biggest challenges was managing computing power. Initially, we used a high-powered computer that required heavy, bulky batteries, making the device difficult to carry. By optimizing algorithms and using highly efficient, cutting-edge processors, we managed to lighten the device, which initially weighed 10 kg but now weighs only 2.5 kg.
F. V.: We also had to solve visual detection issues. For instance, during tasks involving a touchscreen, the camera was disrupted by screen glare, which made it challenging to detect selectable objects. Our engineers, Axel Lansiaux and Éloïse Szmatula, developed technical solutions so that Salomé, our pilot, could complete the task in record time, nearly as quickly as a sighted person.
How did you collaborate with visually impaired pilots to adjust and refine the device according to their specific needs?
L. S.-B.: Working with Sébastien Hinderer, our first pilot, and later with Salomé Nashed completely transformed our understanding of the project and helped us rethink daily life from the perspective of visually impaired individuals. Through conversations with them, we discovered development opportunities we wouldn’t have considered alone. With Salomé, who is energetic and highly driven, we regularly test new tweaks to improve the device. She contributes ideas, and her strong commitment is invaluable.
F. V.: We strongly believe in co-design, where users are involved from the beginning of the development process. Sébastien, and now Salomé, have provided feedback to adjust the harness, refine tactile feedback, shape the handle, and fine-tune audio cues. Salomé and our PhD engineer Éloïse Szmatula have worked together for hours, testing each component, modifying algorithms, and changing training protocols as needed. This ongoing adjustment work has made the technology both high-performing and intuitive. We continue to fine-tune these aspects in preparation for the competition.
How have Sorbonne University students contributed to this project?
L. S.-B.: From the outset, we involved around a hundred master’s students, mainly in robotics, AI, and image processing. Their contribution has been crucial, as approximately 80% of the technical development was accomplished by them. Axel Lansiaux, one of our former students, is now a full-time engineer on the project. He played a key role in supervising several groups of interns and coordinating technical tasks.
F. V.: The students worked very professionally, adopting a client-engineer relationship and following a precise set of requirements. We’re proud to showcase their tangible results in the competition, whether in visual recognition, technical development, or solution integration.
How do you envision the future evolution of this technology?
L. S.-B.: After the competition, our goal is to make our navigation assistance technology open source. The idea is for it to be accessible to all, particularly fab labs, allowing others to replicate and improve it.
F. V.: We also have a PhD project with Eloise Szmatula, who is working on modeling kinesthetic interaction by studying how our system replicates the sensation of being guided by another person. As part of this, we plan to develop a partnership with the National Institute for Blind Youth (INJA) to test our device with young people and their mobility trainers, who will act as guides. This work will allow us to further refine the device, making it even more effective and suitable for daily use by visually impaired individuals.
Interview by Justine Mathieu
Watch the Cybathlon final 2024 with the A-eye team on replay
La finale du Cybathlon 2024 en images
Salomé Nashed lors de la finale du Cybathlon 2024. Photo par ETH Zurich / Cybathlon / Nicola Pitaro
Découvrez l'équipe A-eye
Présentation de l'équipe A-Eye pour le Cybathlon 2024