Environmental Localization Mapping and Guidance for Visual Prosthesis Users (SLAM)

This study is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating Spatial Localization and Mapping (SLAM) and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. The investigators will study how effectively the SLAM technology enables the visual prosthesis system to construct a map of the user's environment and locate the user within that map. The technology will be tested both with normally sighted individuals donning a virtual reality headset and with retinal prosthesis users (Argus II).

About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight Medical Products (SSMP) has developed the world's first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improves users' ability to orient themselves and traverse obstacles, yet lacks features for high-level navigation and semantic interpretation of the surroundings. The proposed study aims to address these limitations of the Argus II through a synergy of state-of-the-art simultaneous localization and mapping (SLAM) and object recognition technologies. This study is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user's environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. Investigators will develop and test a visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies navigationally-relevant objects and landmarks using object recognition and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification.

This intervention will assess the feasibility of using a navigation system to aid blind users of a visual prosthesis with navigation tasks, by using the navigation system to provide navigational cues through multiple sensory modalities including vision and audition. The navigation system will be designed and developed as part of the proposed research and will interface with the Argus II retinal prosthesis system, which is an FDA approved visual prosthesis. Existing blind users of the Argus II device will be recruited for this study, and the navigation system will interface with these subjects' existing Argus II systems. Sighted subjects will also be recruited for this study, in which case the navigation system will interface with a head-mounted display (such as the Oculus Rift) worn by the sighted subjects that simulates the visual sensory information experienced by blind users of the Argus II retinal implant.

We use common psychophysical judgments such as the following: discriminating between two alternatives by saying which is present in the field of view, touching a tablet computer to indicate the apparent location of a target in the frontal plane, and looking for targets by rotating the head and pointing when one is found. We will compare user performance using the Basic system with user performance using the Augmented and Augmented + Modal systems where contextual information associated with the task is conveyed to users via enhanced visual, auditory, and haptic modalities.

The participant will be asked to identify intended targets in a closed environment, as well as to navigate to intended targets. We will measure accuracy as the total number of successes divided by the total number of trials.

This will be assessed by the mean duration of time to identify intended target (in seconds). For successful trials, the shorter the amount of time to conduct a task, the better the performance on said task.

This will be assessed by the metric error in distance judgment, that is, the correctness of identification of judgments of target features. Measurement in meters.

Criteria for inclusion of normally sighted individuals: Subject speaks English; Subjects must be an adult (at least 18 years of age); Subject has the cognitive and communication ability to participate in the study (i.e., follow spoken directions, perform tests, and give feedback); Subject is willing to conduct psychophysics testing up to 4-6 hours per day of testing on 3-5 consecutive days; Subject has visual acuity of 20/40 or better (corrected); Subject is capable of understanding participant information materials and giving written informed consent. Subject is able to walk unassisted Criteria for inclusion of Argus II users: The inclusion criteria for the study are the following: Subject is at least 25 years of age; Subject has been implanted with the Argus II system; Subject's eye has healed from surgery and the surgeon has cleared the subject for programming; Subject has the cognitive and communication ability to participate in the study (i.e., follow spoken directions, perform tests, and give feedback); Subject is willing to conduct psychophysics testing up to 4-6 hours per day of testing on 3-5 consecutive days; Subject is capable of understanding patient information materials and giving written informed consent; Subject is able to walk unassisted. Exclusion criteria for all subjects is the following: Subject is unwilling or unable to travel to testing facility for at least 3 days of testing within a one-week timeframe; Subject does not speak English; Subject has language or hearing impairment.

Results from this study will be disseminated through conference presentations and peer-reviewed publications. However, IPD will not be shared outside the study team.