The focus of this project is to develop an obstacle detection and avoidance system using the Kinect sensor as a vision substitution device.
- The Kinect sensor is chosen due to its ability to capture depth data along with the RGB channel, which can help in constructing a 3D map of the user’s field of view and aid in navigation.
- Additionally, it is also equipped with an infrared camera that can be used to enable night vision, providing unhindered performance in both light and dark environments.
- The proposed system will also be integrated with a facial recognition and emotion detection module, allowing it to recognize familiar faces in an environment and capture non-verbal cues such as smiles during a conversation, thereby adding some social benefit.
- Object detection results, Navigational directions and facial analysis results will be communicated back to the user via an auditory channel in real-time.
The frame capture of the sensor runs in a loop that is triggered by an init signal. This signal triggers the image acquisition module which captures the appropriate type of image (RGB/IR) along with the depth map. This data, along with the user provided control signal is sent to the core module which executes the appropriate function and outputs the corresponding result. This result, which is in text format is converted to audio and conveyed back to the user.
The hardware for this project involves the Kinect sensor. It consists of an RGB camera that produces images at 640x480 pixels, as well as a depth sensing system, that consists of an IR laser emitter and an IR camera, and produces images at 640x480 pixels. The depth measurement is done using the infrared emitter and camera whereas the computation is done using a patented structured light technique [25]. The field of view of the system is 58 degrees horizontal, 45 degrees vertical, 70 degrees diagonal, and the operational range is between 0.8 meters (2.6 ft) and 3.5 meters (11 ft), both of which is determined by the sensor. The frame rate of this sensor is 30 FPS.
