"It's like Shazam for the world around you." -Fast Company
What is in the video is BlindTool V1. I want to build BlindTool V2!
What does BlindTool V1 do?
Advances in computer vision research have made it possible for a phone to see! This app tells you what it is looking at and vibrates based on how confident it is. When using this app you will wave the phone around you until you feel it vibrating more and more which means you are getting closer to an object it understands. The convolutional neural network inside the app can understand 1000 labels based on the ImageNet dataset.
+Runs completely on the phone without internet
+Vibrates when confident about prediction
+Understands 1000 things
+It is available now and it is free!
I want to make BlindTool into a tool that the visually impaired use everyday. Currently there are problems that need to be overcome:
-Does not understand many household objects and concepts (e.g. wall)
-Is inaccurate and confused by some objects
-It doesn't turn on the flashlight when it is dark
-It doesn't have any options to customize it
What will BlindTool V2 do?
While BlindTool represents a breakthrough in computer vision technology in the quality and speed of the neural network that is able to run on a phone there are still ways to make the system better.
+It will still be free!
+Gather labels and images from users of what they want to have the system see and then train the system to identify these things. (Training cannot happen on the phone)
+Add a way to customize the systems interaction by filtering out some labels or add different names for identified objects.
+Test the app more with blind users to make sure it is something they want to use everyday
+Add a flashlight option with proper UI so the battery won't be drained if it is left on
+Add translation to work with other languages
Risks and challenges
Sometimes with neural networks, specific labels can be very challenging but as a researcher I believe I can adjust the network to solve these problems. I have many research publications relating to image recognition tasks.
When training from images, a network can take from days to weeks. I will have to acquire a better graphics card in order to train the network.Learn about accountability on Kickstarter
- (30 days)