This entry is for app creation targeting Tablets.
Sagen will be listed under Health Care related apps category.
"The saddest thing in life is wasted talent." - A Bronx Talent.
But even then, there are millions of people suffering from different forms of autism and speech impairment around the world. It is really sad to see so much potential being wasted as millions of people are unable to lead a normal life. Through Sagen it is my endeavour to make lives of people suffering from speech impairment easier by arming them with this app which will help them speak, While it is true that this app is no replacement for people who are suffering from different forms of speech impairment but it would be a small step forward in solving this problem through the power of technology.
Sagen, is a german word that stand for "to speak" and this is exactly what Sagen as an app will do. Sagen will help people who are speech impaired, by actually speaking for them. A person using Sagen would be able to directly tell Sagen what they want to speak and Sagen will actually say that for them.
How will Sagen do this?
The basic idea is that you tell Sagen what you have to say, and it will say that for you. The best way to tell Sagen what you want to say is by typing your response into Sagen and then, Sagen will automatically speak those words through the speakers. Sagen will also offer helpful suggestions based on the current context and your previous history of responses.
Sagen will also have some default responses stored in it. These responses will be those which are most common responses to most common situations. Sagen will learn as you use it. If there's a sentence that you often say, Sagen will learn that and it will offer to speak it for you next time when a certain situation arises,
By making use of the categories, Sagen will segregate and differentiate these sentences that are stored by default. For example, we can have a category like "Doing Homework" that will help a child to a category like "Baking" that is more adult. Each category will have certain specific statements of their own and one can of course, coin their own phrases. It would become clearer by looking at the screenshot nearing the bottom of the article.
Arguably, the best part of Sagen will be its ability to predict what the user is going to say next. Sagen will do this by appropriately considering the person's geographic location, demographics, speaking idiosyncrasies of the people living around that particular area when processing the Natural Language for a certain response. To a certain question, Sagen will show you the appropriate responses, by judging what it thinks is right. Basic responses will be built right into Sagen for offline use and for more serious tasks Sagen will need to connect to the internet. There are many things that one has to take into consideration while developing something like this and going into details of all of them is outside the scope of this article.
Admittedly, Sagen works on a fifteen year technology which is speech processing and recognition but Sagen adds so much on top of all the work that has been done. I wanted to develop a system of speech that would feel natural and the users of which will have ease in communicating. By leveraging the power of cloud, Sagen will become something that wasn't possible 15 years ago, when speech recognition came into being.
Approximately 7.5 million people in the United States alone who have trouble using their voices. If we take the whole world in the picture, there's no denying that Sagen will have a huge impact on the world. Sagen gives the power of speech into hands of those who are unfortunate enough not to have it. The real impact of Sagen will of course be seen only after it reaches into the hands of those that are the target audience of Sagen.
Working with an NGO, I can help Sagen reach it targeted audience of people who have speech impediment as well as gauge and monitor their performance as they go about their daily tasks with the help of Sagen. I'm already in talks with a couple of NGO's based here in New Delhi and they are really excited to test and take Sagen to the next level. NGO's will also serve as a good testbed for Sagen as I would be able to test Sagen in the field getting real-time data and building upon it.
Living in India and owing to some of my own personal experience, I am very close to the plight of the people who are less fortunate than the most of us. It is for this reason that I'm always trying to find some way that will help relieve these people from their disability. For me, technology is about making powerful impact and creating social change in the world.
I got the idea about this app while playing around with the API of Bing Translate. I was working on an app that would automatically translates the sentences being spoken and help you determine the appropriate response. In the same way that language translation breaks down barriers between people of different regions altogether, I believe that Sagen will open new doors for the people who have some sort of speech impediment.
The natural language prediction will be based on fully parametric statistical inference algorithm models that works best with an unfamiliar input. This is really important because everyday since Sagen will come across sentences that it has never come across before and it has to make a sense of all those sentences and find the nuances in the language. This is because we are assuming that all of the inputs can be fully described by a family of probability distributions involving only a very limited number of unknown parameters.
Sagen is a desktop app to be developed using C# and XAML in Silverlight 5. The speech recognition will be performed on the client machine using Silverlight itself using the speech API of Silverlight.
After development of Sagen, I'd want it to support as many languages as possible besides English. This would be another herculean task not because of the sheer amount of languages present in the world but because of the amount of data processing that it would need to process such huge amounts of data. But we'll think about it when we get to that.
Points of Interest
- What makes Sagen so great for a tablet is because a tablet has more screen real estate than a traditional smartphone. Moreover, a tablet also has a higher processing power than a smartphone making it an ideal choice for Natural Language Processing.
- One more way that Sagen can work is when you're in a foreign country and you don't know the language. By making use of the different languages and translating between them, Sagen can help you determine what the other person is trying to say and what your natural response can and should be, in their foreign language,
Mockup of food category in Sagen
Writing and Speaking sentences through Sagen
- Update 2: Made many changes to the UI but it's still a work in progress, Started working on implementing the NLP through Windows Azure.
- Update 1: Completed the rough interface of Sagen and the integration of Bing API.