For people with blindness or who have low vision, moving around independently is a daily challenge. Harvard researchers have developed a new tool that could help such people navigate more quickly and safely, using nothing but a device they likely already have – a smartphone. Engineers at the John A. Paulson School of Engineering and Applied Sciences (SEAS) have invented a new smartphone app called Mobilio that combines machine learning; algorithms that streamline input from multiple sensors; and personalized audio cues to provide a blind or low-vision person turn-by-turn directions, path guidance, and obstacle avoidance. The research was led by Ph.D. student Raymond Liu in collaboration with Patrick Slade, assistant professor of bioengineering. The research is published in Nature Biomedical Engineering. To start, the researchers surveyed over 100 blind or low-vision people to understand their needs. Respondents highlighted three essential capabilities for navigation technologies: reliable turn-by-turn directions, continuous guidance along sidewalks and paths, and obstacle detection and avoidance. Existing tools like canes, guide dogs, and electronic travel aids address parts of these problems but rarely all three together, and many such systems are expensive and inaccessible. While GPS map software has turn-by-turn navigation, it is not accurate enough for a person who is blind or low-vision to safely use. By contrast, most people already have smartphones. The Harvard team saw an opportunity to deliver a navigation aid through a standalone app that could one day be downloaded from an app store and runs entirely on the phone. Mobilio utilizes the phone’s built-in sensors, including the camera; GPS; motion- and orientation-sensitive inertial measurement unit; and (if available) the LiDAR sensor. A central technological advance is a custom computer-vision model that analyzes the live camera feed from the perspective of a pedestrian to find walkable paths in the user’s environment. “Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” Slade said.