Read For Blind is an app that lets blind people read any standard text ( books, visiting cards etc) at their own without any assistance. The blind people is expected to perform just the basic task of starting and closing the app. Rest of the time, he has to simply follow the app's instruction. The app captures an image of the text and reads it out. This is done by asking the blind person to move the device in different directions until the whole text is in camera view. My team has already implemented the idea for Windows Phone 8. This app has been tested on actual blind people, here is a link to show the app in action - App Testing Video
- The app is for blind people so it is totally voice controlled.
- The user can start, stop or give any other command to the app using the specified voice commands.
- The blind person starts the app with voice command and places the device above the text that he wants to read( the blind person can sense where the document is).
- Our app automatically detects the text and gives instruction to the user to move the device left, right, top or bottom. The app itself captures an image of the text when all the text on the page is in the field of view of the camera.
- Once the image has been captured, text is extracted from the image and read out to the user.
- The app also has play, restart, repeat options.
- The feature of translation can also be included.
The whole development procedure can be divided into three parts :
- Image capture at appropriate time to ensure that all the text has been detected and image processing to use a clean image for OCR.
- Optical Character Recognition to extract the text from the captured image.
- Implementing the voice commands to ensure the smooth navigation for the blind person.
Conceptualizing the App :
- Coding Language : C#
- OCR : Project Hawaii
- Voice commands : SSML
Please note that this app has already been developed for Windows Phone 8 platform, so developing it for Tablet will not be a very difficult task.
Ongoing Development Challenges
I started the app development for Windows 8 Tablet expecting all the capabilities of the Windows Phone to be available but unfortunately it is not so. The app involves fair bit of image processing in real time. For this, I have sifted to OpenCV but it took a long time to get OpenCV working for .Net Platform. As of now, I have implemented Text-To-Speech and also the OCR part. Next thing to be implemented is image capture feature. It needs some level of sophistication as my target user will be a blind person.
I am an Undergraduate student currently in my 3rd year of B.Tech in Information and Communication Technology at DA-IICT, Gandhinagar. I am very passionate about application of technology for social welfare. I have been working in a team since my first year of undergraduate.
Software Development history :
- I developed a basic app for I Unlock Joy program of Microsoft, during the first year of my B.Tech.
- I worked with one of my friends to develop a game for Microsoft's Dream Build and Play contest. Here is a link depicting the basic idea : https://plus.google.com/u/0/118436473022163490185/posts
- I developed Read For Blind in a team of four for Imagine Cup 2013. Here is a link to show the app in action : http://www.youtube.com/watch?v=6gbXtcemQ5k