This project is a proof of concept. I originally had this idea to make a device to assist people who are blind. I figured I could use the Kinects depth camera, computer vision, and Arduino to send the depth information down to a vibrating-tactile display. If this tactile display were small enough it could fit in the palm of your hand and could be used to essentially feel the objects and people around you.

    As an example, if you are having a discussion with a friend they are usually significantly closer to you than the objects in the background like walls, chairs, buildings, etc. As a result your friends silhouette is easily detected with depth information from the Kinect. Their silhouette can then be transformed into a tactile sensation, giving the user a better understanding of where their friend is in relation to themselves. This would also work for object detection or object avoidance when navigating through a room or crossing a busy intersection.

    The video bellow demonstrates my working prototype. A friend of mine walks back and forth in my apartment, and with my eyes closed I am able to literally feel where in the room she is. The video at the bottom↓ of this page does a great job of explaining how the prototype actually works.



    Bellow are two images of my working prototype. The image on the left shows the hard-foam housing for the vibrating motors which allowed them to vibrate freely while keeping them supported, and the circuit board I assembled to interface with the Arduino.  The circuit board was designed by my professors Tony Olsson and David Cuartielles for 1Scale1 which provided 64 multiplexed pins with an easy to use library for Arduino. The image on the right shows its size in relation to the hand and fingers.  The vibrating motors are arranged to stick up about 2mm above the white hard-foam to improve the tactile feedback felt by the hand.



    Bellow is a video of my first working prototype. Looking back on it, its actually a great way to explain how my prototype works to sighted-people. The upper right window is the regular color video from the Kinect. The upper left window is the depth video, where nearby objects are light gray and far objects are dark gray. The bottom window is a grid of 8x8 LEDs controlled by an Arduino.

    As I move back and forth one can see little white boxes appear in the upper left window over my silhouette. These boxes are arranged in a grid of 8x8, which correspond to the grid of 8x8 LEDs. So as I move around, the computer detects my silhouette, puts a white box over it, and then turns on the corresponding LED in the matrix bellow.

    One thing that can clearly be seen in this video is that 8x8 is too low of a resolution for anything more than a working prototype. I would guess a 16x16 or 32x32 grid would work better for doing things like user testing.