Would it be theoretically possible to take eye tracking (gaze position) data and feed it into some kind of wearable device/virtual reality setup that would cause the whole body or parts of the body to feel as if they are moving in a direction that's counter to how the eyes are moving? If so, what kind of technology would be the best way to do this?
It would have to rely mainly on haptic feedback, as this is to help blind people with very limited light perception. Hopefully a visual component would work as well, but it would have to be very simple, like a moving dot.
The feeling of movement when a virtual scene rapidly moved is really strong and can sometimes even lead to injuries because a person tries to counter a perceived movement while he actually stands perfectly still. This is a big problem for VR applications: The screen suddenly freezes while the person using it is moving the head. The screen is stopped and the person involved perceives this as strong movement against the original movement/rotation.
If you can work some visual cues, you could just use a large white spot or orb in a dark scene and move it to the left and right according to your eyetracking input.
I can also imagine that using sound would work aswell. You could use a stereo headphone and translate the information of movement direction into a syntezied tone. I imagine that varying the amplitude of the sound is good for absolute position infromation and the pitch of the tone for encoding speed information.
When the person in the setup moved, he will perceives his virtual motion as real motion because the sound source will move as it would in the real space.
Depending on your VR hardware, a prototype should be realizable in only a few days. I would use Unity3D and hook a moveable sound source to the eyetracker (for example Tobii Eyetracker for HTC Vive). In theory you only need the eyetracker for this to work and the VR setup is optional.
I'm not sure the sound will work; I think muscles will need to be electrically stimulated for what I'm going for? I want the user to feel as if his or her body is tilting or rotating in a direction counter to eye movements.
A couple more questions:
1. Would it be possible to use electronic muscle stimulation to isolate the eye-body link to specific body parts (i.e. one arm)? I know some people at MIT have done something like this with their Tree Sense project, so I'm thinking yes?
2. The purpose of what I'm trying to do is to use proprioception and motion to teach blind people about visual concepts and improve spatial perception abilities. I'm blind myself and I know that focusing on the relationship between the eyes and other body parts has helped me form a map of space, but I think technology can enhance this link. Would anyone be interested in working with me more on this, or hearing more about it?
Tasha,
I would greatly appreciate the opportunity to hear more about this project AND to work with you. I have two eye-trackers (e.g., Gazepoint and Tobii 120). I am also a nationally Certified Adapted Physical Educator (CAPE) and have worked at several CAMP Abilities across the United States (sport camps for children with visual impairments, blindness, and deaf-blind). Please let me know how I can help you! Thank you so much!
All the best,
Joann
www.campabilities.org
Wow, the fact that you are a physical educator might really help with this because the sensation I'm trying to replicate feels very much like climbing, or working out on a weight machine, because oppositional force is involved. Feel free to email me at [email protected]. I can give you more background (please feel fre to check out my paper, "Mending the mirror as well; it's on here), and if you're interested, we can set up a phone call.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers