Share this project

Done

Share this project

Done
Help our neuroscientific project to expand human perception. We're making a wearable device to let brains take in new data streams.
Help our neuroscientific project to expand human perception. We're making a wearable device to let brains take in new data streams.
288 backers pledged $47,285 to help bring this project to life.

About this project

VEST: A Sensory Substitution Neuroscience Project project video thumbnail
Replay with sound
Play with
sound

$47,285

288

Can brain science + several little motors + a cell phone expand the limits of our biology?

I'm Dr. David Eagleman, a neuroscientist at Baylor College of Medicine in Houston, Texas. My graduate student Scott Novich and I are on a mission to see how far we can push the limits of human perception, and what it means to be human in general. We want to invite you along for the ride!

We're undertaking an audacious task: developing the most advanced sensory substitution system to-date. We're building a wearable vest that communicates sound to the brain using the sense of touch.  We expect this will be powerful enough to give deaf individuals a new "sense" of hearing. We call it the Versatile Extra-Sensory Transducer, or VEST.

Sound is captured and processed on a smartphone. The data is then sent over Bluetooth and played in real-time using a series of vibration motors on on a wearable vest.
Sound is captured and processed on a smartphone. The data is then sent over Bluetooth and played in real-time using a series of vibration motors on on a wearable vest.

 The idea of sensory substitution is not new: it was pioneered by Paul Bach-y-Rita in 1969 with blind participants. He developed a dental chair with an array of push pins on its back, which was attached to a video-camera feed. Blind participants sat in this chair, and felt what was presented in front of the camera. After practice, the participants began to develop a visual intuition for the sensations they felt. Today, the current incarnation of this device is called the Brainport, and blind individuals have been able to use this in complex visual tasks (like obstacle course navigation).

Our goal is to leverage the promise of sensory substitution in the realm of hearing, using the latest algorithms and hardware. We've developed a working prototype and have great preliminary data:

Our results from a word identification experiment demonstrate that participants (hearing participants, in this case) can learn to interpret sound with the vest.
Our results from a word identification experiment demonstrate that participants (hearing participants, in this case) can learn to interpret sound with the vest.

We hypothesize that our device will work for deaf individuals, and even be good enough to provide a new perception of hearing. This itself has a number of societal benefits: such a device would cost orders of magnitude less than cochlear implants (hundreds-to-thousands as a opposed to tens-of-thousands), be discrete, and give the wearer the freedom to not be attached to it all the time. The cost effectiveness of the device would also make it realistic to distribute it widely in developing countries.

More exciting than this, however, is what this proof of principle might enable: the ability to feed all sorts of new and profound sensory information into our brains. Anything from Twitter, stock markets, the weather, and beyond.

Why we want your help

We need your help in taking this study to completion, and are excited about inviting you to share the ride of science. Funds raised will go towards a wide range of costs associated with this endeavor, including:

  • Prototype design and refinement costs (from purchasing components to manufacturing circuit boards)
  • Funding graduate student salaries for this research (help smart guys like Scott!)

Our research

Visit the website for this project (eaglemanlab.net/sensory-substitution)! We will continually post details and updates here. We are also happy to answer questions on this Kickstarter page about our proposed research plan:

Step I: Prototype Refinement (The hard work!)

We're developing and experimenting to determine the best method of compressing and translating audio so it can be understood using vibrations on the skin. We have engineered several candidate algorithms to be tested and compared against one another. We are also working to optimize the layout of motors on the vest: what is the best mapping between the sensory auditory information and the spatial location and vibration patterns of the vibratory motors?

Step II: Behavioral Testing (The Exciting Part!)

Once we fine-tune the prototype and training procedure, the next step is to quantify how well the device works with deaf participants over a range of ages and types of hearing loss. These results will be published in scientific journals and will be available to you!

Step III: Brain Imaging Studies (Another Exciting Part!)

We'll study the brain mechanisms behind this kind of learning by imaging the brains of participants as they move through training using functional magnetic resonance imaging and diffusion tensor imaging. Again, our results will be published in scientific journals and will be available to you

Current prototype Q&A

  • How long does someone need to wear the VEST to be able to understand something? Our preliminary data hints that leaning to interpret audio likely takes on the order of 2-3 weeks. We hope to gather better estimates and answers thanks to your contributions for this project!
  • What are the limits? For audio, it's a matter of the quality of our algorithms, which are designed around the constraints of our sense of touch. Our best algorithms at the moment are able to capture enough information that speech and basic environmental sounds are intelligible. Musical enjoyment is not happening at the moment, but this always has the potential to change!
  • What can it do now? The current prototype is fully usable with real-time processing capabilities and is portable. The effectiveness of the device and our algorithms with long-term usage will be measured thanks to your contributions!
  •  What does it feel like? Does it hurt or chafe? The device is comfortable enough to wear for extended periods. We'll be making it even more comfortable as we move forward.
  • How heavy is it? The Vest currently weighs about 10 lbs (4-5 kg). This weight can also be cut down significantly in future iterations.
  • Does this device work for all deaf individuals? This is what we hope to find out with your support! Whether someone is young or old, deaf since birth or deaf as an adult, science will answer these questions for us!

Giving back

We have an assortment of prizes as a way of showing our gratitude for your support of this project. Our graphic-designer extraordinaire friend Ben Mautner has created a stylish VEST logo (the bars are an artistic rendition of the formants of someone saying "vest"), which we're prepared to print up on t-shirts as one of our rewards:

Example shirt. Final color-scheme may differ!
Example shirt. Final color-scheme may differ!

At higher funding levels, we have rewards ranging from signed copies of my books, some one-on-one Google Hangout time, and even getting your very own vest to experiment with.

Highlights/Notes on some of the reward levels:

1. Explorer level ($10): We'll give you an app (Android only) that lets you listen to the audible results of our algorithms in real-time. And we'll send updates so you can track our progress! While the audio we decode may not reflect the actual perceptions one may develop by wearing the VEST, this is one of the diagnostic tools we use in our lab to determine if our algorithms sound good. It is recommended you have a more recent Android device. 

2. Hacker level ($25): We'd love to spur others to innovate in this area. We'll help you get started with some of the core functionality that is needed for implementing a real-time sensory substitution device using an Android and Arduino framework. We'll give you well-commented tutorial projects that (i) demonstrate setting up Bluetooth communication between Arduino and Android, (ii) perform real-time audio processing on Android, and (iii) linking both concepts together!  This being said, we're not commercial developers, so we make no claims about the optimality or quality of our code. We just know it works!

3. Sense-sationalist ($3000) and Tactile-technician ($3500) levels: We'll wire up and ship you your very own VEST! And we'll send you app-updates as our research progresses. We'll also send you our sound-training apps so you can be a remote participant if you'd like!

Some important notes: (i) You will need to supply your own up-to-date Android tablet or phone (running Ice Cream Sandwich or newer), (ii) The hardware you will receive is in no way a finished product and (ii) we make no warranties on how long it lasts, how well it will work, or how safe it is.

Clear instructions will be provided on how to operate the device. The device will be tested to ensure it is fully operational before being shipped, and troubleshooting applications will be provided. We will do our best to help with fixes and troubleshooting if the Vest breaks at your end.  

Video credits

We couldn't have done this without help from Ryan Booth (@ryanbooth) and Henry Proegler (@henryproegler). Many thanks to the both of them for producing our Kickstarter video!

Risks and challenges

This is primarily a *science* project that involves some cool hardware. Our hypothesis, which has been carefully formulated by previous scientific work and years of research, is that deaf individuals can come to gain a new perception of audition using our device. As with any scientific hypothesis, a negative outcome is always a possibility, yet even a negative outcome still counts as scientific progress. One thing is for certain: we'll definitely discover a lot about the flexibility of the brain along the way. Thank you for helping us on the journey!

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support this project

  1. Select this reward

    Pledge $5 or more About $5

    Comrade of science level: We'll list you as a donor on our website, and keep you updated each quarter with progress.

    Less
    Estimated delivery
    62 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge $10 or more About $10

    Explorer level: Wonder what the information we're capturing sounds like after it's been processed by our algorithms? In addition to the previous level, get our real-time audio-processing Android app and hear how the world might sound through the vest! We'll send out updates as our algorithms are further refined!

    Less
    Estimated delivery
    39 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge $25 or more About $25

    Hacker level: This one goes out to the makers+hackers! In addition to the previous award levels, we'll drop you Android and Arduino source-code that shows you how to do both real-time audio processing and Bluetooth transmission/reception between Android and Arduino.

    Less
    Estimated delivery
    46 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge $40 or more About $40

    Science hotshot level: You're really allowing the science to happen! Get your very own super fly V.E.S.T. t-shirt.

    Less
    Estimated delivery
    Ships to Only United States
    26 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge $48 or more About $48

    Science hotshot level (****International Edition****): We've now added an international shipping option! You're really allowing the science to happen! Get your very own super fly V.E.S.T. t-shirt. [See http://goo.gl/IYTWNR for available countries]

    Less
    Estimated delivery
    Ships to Anywhere in the world
    5 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge $50 or more About $50

    Science superhero: Get a V.E.S.T. t-shirt and get personally thanked (and followed) by David on Twitter.

    Less
    Estimated delivery
    Ships to Only United States
    21 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  7. Select this reward

    Pledge $58 or more About $58

    Science superhero (****International Edition****): Get a V.E.S.T. t-shirt and get personally thanked (and followed) by David on Twitter. [See http://goo.gl/IYTWNR for available countries]

    Less
    Estimated delivery
    Ships to Anywhere in the world
    3 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  8. Select this reward

    Pledge $75 or more About $75

    Science supah-stah: Get a t-shirt, a personal thank you tweet from David, and a 10 minute Google Hangout session with David and/or Scott (your choice)!

    Less
    Estimated delivery
    Ships to Only United States
    Limited 8 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  9. Select this reward

    Pledge $100 or more About $100

    Enlightened enthusiast level: Beyond the above accolades and incredible appreciation, get a signed advance copy of David's next book, "Livewired."

    Less
    Estimated delivery
    Ships to Only United States
    Limited 29 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  10. Select this reward

    Pledge $250 or more About $250

    Sensory scholar level: You're making us cry with joy. In addition to the previous level rewards, get signed copies of all of David's books: "Livewired," "Incognito," "Sum," and "Wednesday is Indigo Blue."

    Less
    Estimated delivery
    Ships to Only United States
    Limited 5 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  11. Select this reward

    Pledge $300 or more About $300

    Excitatory neuron level: Our science is now cooking with gas! In addition to all of the previous level rewards, get your name in big letters on a prominent section of the Eagleman Lab wall for a year!

    Less
    Estimated delivery
    Ships to Only United States
    Limited 3 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  12. Select this reward

    Pledge $1,000 or more About $1,000

    Science shmoozer level: In addition to all of the previous levels, get a 1-hour 1-on-1 Google Hangout with David and/or Scott (your choice). Ask us questions about neuroscience and engineering!

    Less
    Estimated delivery
    Ships to Only United States
    Limited 4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  13. Select this reward

    Pledge $3,000 or more About $3,000

    Sense-sationalist level: Get your own V.E.S.T. and the sound-to-touch processing application for Android. Previously mentioned autographed books, app, and t-shirt included.

    Less
    Estimated delivery
    Ships to Only United States
    Limited 0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  14. Reward no longer available

    Pledge $3,500 or more About $3,500

    Tactile technician level: Get your own V.E.S.T. (no air-quotes!), the sound-to-touch processing application for Android, and access to developer libraries and tools! Develop your own tactile sensory augmentation vest! Previously mentioned autographed books, app, and t-shirt included.

    Less
    Estimated delivery
    Ships to Only United States
    Reward no longer available 3 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  15. Select this reward

    Pledge $10,000 About $10,000

    Elite scientist level: David will give you and any number of your guests a 1 hour private lecture (continental US only). Previously mentioned autographed books, app, and t-shirt included.

    Less
    Estimated delivery
    Limited 1 backer
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (30 days)