Click here to Skip to main content
15,435,482 members
Articles / Programming Languages / Java
Posted 10 Sep 2015

Tagged as


13 bookmarked

Artificial Intelligent Home Assistant

Rate me:
Please Sign up or sign in to vote.
4.83/5 (8 votes)
10 Sep 2015CPOL6 min read
An Artificial Intelligent Home Assistant


In this article, I will be discussing about autonomous intelligent Home Assistant. It is a virtual Robot for you and your home. It will take care of all your home needs. It will check your household groceries and order for you. It will speak to you and you can speak to the virtual robot. The robot will be able to recognize your commands and order whatever products you want to order.

It is helpful for all families in bringing the intelligent systems in their lives. There is no need to go to shopping malls and it is a new way of ordering things. The intelligent desktop app will do all the things for you. You can order directly or else it will order intelligently according to your needs.

App Specifications

You may have heard about Richard Wallace Chat Bot, the AIML based chatting application. The same AIML (Artificial Intelligence Markup Language) technology is used for background intelligence of virtual Robot. The App will have a speaking robot as GUI and also it can work as a background task. It will use Google's speech engine for converting your speech into Text or Microsoft speech Engine. The AIML and WebServices are used for taking your order. A wide angle bar code scanner which can scan 4 to 5 items at a time can be used to scan the predefined tags on grocery bottles. The motor connected to scanner will scan all the grocery bottles and identify the quantity of product is less than 20 % or not. The scanner is Microsoft Kinect sensor itself. Whenever your household items especially groceries are less than 20%, then it will automatically order for the new products.

Remaining Products

There are many other products which are quantity based like toothpaste, hair oil, facial creams, soaps and perfumes which are mostly constant on a monthly basis. It will order for you every month depending on your usage. And also, if you need to order something else, you can order directly by your voice or by using scanning. Daily orders like milk, curry leaves, and other items will be ordered on a daily basis.

Example for Voice Order

The voice order will be processed like this example if you need a cold drink.

Human: Get me a cold drink

Robot: I am ordering for you a Coke from Walmart. Is it ok?

Explanation: The process will happen like this. It will take cold drink as a keyword from your sentence and it will pick your most frequent order, i:e., Coke in this example and order from the nearest retailer shop Walmart as it is the nearest store and it will deduct from your account and place an order. Then if you say yes or ok, it will get it for you, or else, it will ask questions depending on your criteria and the order will be made.

Also, if you want to order a particular item from some particular shop, then you are welcome.

Human: Get me pepsi from Reliance Mart.

Robot: Yes Sir, should I order?

Depending on your needs, it will constantly be monitoring you and ordering for you and make you feel happy.

Some Valuables

If you need some other items like dress or jewellery, you can dress virtually using Kinect sensor. It will measure your body measurements and give you a variety of options to choose from and also change colors accordingly if you fit into the clothes. You can virtually wear jewellery and check from home itself. This is new style of shopping the things. The Artificial intelligence and Powerful Computing Machine will give you a new style of living. You can even monitor your home or children from your office by pinging to your robot address and you can virtually play with children and look after them.

When any product is completed, you have a chance to renew new things by making voice commands or else scan and throw your waste wrappers it will order for you.

About the Uploaded Files

The uploaded files are an intelligent chat system image, and a Robot which can stream video and audio and also it will have intelligent speech with humans which I am currently developing.

Category and Platform Details

My category goes into Retail (the way we shop). According to category, my platform goes to the tablet. But since I have to do video processing and Kinect barcode scanning, I think the platform should be All in One (for good amount of run time memory). So I request for All in One.


  • The first step is to scan bar code using kine-ct sensor and get the bar code and process it to get the details on the bar code.
    • Then from the back end, the submission form of order will be filled with defaults depending on the users interest (He usually takes 1kilogram quantity from Walmart store). The form will be filled accordingly and the order will be processed.
    • And whatever is going on should be notified to the user at billing time.
  • The second step is getting voice commands from the user. Every time, it will listen to you depending on the amplitude of your voice.
    • If you are in conversation with others, the application will listen for 10 seconds and it will delete the conversation.
    • If you want to start a conversation with the robot, you can ping it in two ways. Come into field view of robot and speak to robot, or you can start making a conversation by specifying it like "Robot what are you doing?"
    • Then robot will be taken as keyword and it will give an answer according to your question.
  • The third step is to order taking from your commands. You say robo get me ice cream.
    • The Robot will see the pattern of the sentence if pattern like "get me ***" take the star as keyword and search in the product list and its manufacturing list and then retailers list and order for you.
    • It will order chocolate ice cream from vanilla brand from nearby retail store, and that toi depends on users' previous orders.
  • The fourth step is video streaming remotely. Talk to your robot remotely and monitor your house and also through video streaming, you can entertain your children.

Coding Languages

  • AIML
  • C#
  • Kinect SDK
  • XML
  • JavaScript
  • Zxing library for bar code scanning
  • and other utils

Currently, I am working on AndroHumanoid Robot which is a replica of virtual home assistant. Now, I am working on real robot. The same robot which can be made virtually for intelligent homes as discussed above.

You can watch the video below:


  • 11th September, 2015: Initial version


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By
Software Developer self employed
India India
I am android and Robotics developer. I love to do electronic projects. I can summarize my self in three words.
concise,creative and compelling.

Comments and Discussions

QuestionHow's app development going? Will you be submitting on time? Pin
Kevin Priddle22-Oct-13 10:34
professionalKevin Priddle22-Oct-13 10:34 
AnswerRe: How's app development going? Will you be submitting on time? Pin
Gnanendra Kumar22-Oct-13 17:23
professionalGnanendra Kumar22-Oct-13 17:23 
QuestionYou lost me at 'u'. Pin
DanielSheets29-Aug-13 9:23
MemberDanielSheets29-Aug-13 9:23 
AnswerRe: You lost me at 'u'. Pin
Gnanendra Kumar29-Aug-13 17:46
professionalGnanendra Kumar29-Aug-13 17:46 
GeneralRe: You lost me at 'u'. Pin
DanielSheets30-Aug-13 1:14
MemberDanielSheets30-Aug-13 1:14 
GeneralMy vote of 5 Pin
ThatsAlok31-Jul-13 23:55
MemberThatsAlok31-Jul-13 23:55 
GeneralRe: My vote of 5 Pin
Gnanendra Kumar1-Aug-13 0:06
professionalGnanendra Kumar1-Aug-13 0:06 
Questionreduce the width of chat image Pin
ThatsAlok31-Jul-13 23:55
MemberThatsAlok31-Jul-13 23:55 
AdminThanks for your submission! Pin
Kevin Priddle31-Jul-13 7:54
professionalKevin Priddle31-Jul-13 7:54 
AdminRe: Thanks for your submission! Pin
Kevin Priddle2-Aug-13 3:52
professionalKevin Priddle2-Aug-13 3:52 
GeneralRe: Thanks for your submission! Pin
Gnanendra Kumar2-Aug-13 5:20
professionalGnanendra Kumar2-Aug-13 5:20 
GeneralRe: Thanks for your submission! Pin
Kevin Priddle2-Aug-13 5:39
professionalKevin Priddle2-Aug-13 5:39 
GeneralMessage Closed Pin
27-Jul-13 21:44
Memberuser558320827-Jul-13 21:44 
GeneralRe: My vote of 5 Pin
Gnanendra Kumar28-Jul-13 6:36
professionalGnanendra Kumar28-Jul-13 6:36 
GeneralMessage Closed Pin
28-Jul-13 10:22
Memberuser558320828-Jul-13 10:22 
GeneralRe: My vote of 5 Pin
Gnanendra Kumar31-Jul-13 23:05
professionalGnanendra Kumar31-Jul-13 23:05 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.