Click here to Skip to main content
15,861,168 members
Articles / Web Development / HTML

Home Automation with Netduino and Kinect

Rate me:
Please Sign up or sign in to vote.
4.97/5 (159 votes)
3 Jan 2015CPOL5 min read 423.8K   2.1K   355   122
Remote control a squirt gun, open the garage, and water the garden all through .net

Introduction

Home automation has been an interest of mine for a long time. There is a bunch of bad technology in the marketplace and the products are too expensive so I decided to build my own. I started out with the Arduino microcontroller which was really fun but the code quickly became hard to maintain because it was not object oriented. Additionally it could not do multithreading or real debugging with breakpoints and such. I refactored the code for C# and the .NET Micro Framework. I choose the netduino plus, http://www.netduino.com/netduinoplus/specs.htm, for the microcontroller which has a built in Ethernet adapter for network communication.

Please be sure to also see the sequels to this article:

The figure below shows an early prototype of the project.

NetduinoHomeAutomation/LogicalDan_001.jpg

Netduino Controlled Squirt Gun

The first project that I built was a servo controlled squirt gun for the pool. The code that I wrote for the netduino controls the servos to spray the gun in different patterns in the pool. I then built a Windows Phone 7 interface to aim the servos to the position on the screen where you touch. I used IIS live smooth streaming to stream video to the phone so you could remotely nail the kids in the pool from anywhere. I had mixed results with the video piece and at some point I need to spend more time perfecting and reducing the buffering time to make it more real time.

NetduinoHomeAutomation/LogicalDan_002.jpg NetduinoHomeAutomation/LogicalDan_003.jpg

Garden

My next project was to control the irrigation of the garden. My code schedules the times to water the garden and controls the duration of the watering.

NetduinoHomeAutomation/LogicalDan_004.jpg

Kinect

One of my colleagues in the office started doing projects with the Microsoft Kinect which has a rich SDK complete with drivers, APIs and plenty of good sample code. The Kinect has a bunch of sensors including a RGB camera, depth sensor and multi-array microphone. With a Kinect, you are the controller! I got the idea of using the Kinect for the controller of the squirt gun in the pool. You can now aim the gun by pointing to where you want it to shoot. The trigger is controlled by bending your other arm so that the hand is above the elbow joint. Plugging in the Kinect for the controller was really simple because of the rich Kinect API and because I had already written the back end tiers to communicate with the netduino microcontroller.

NetduinoHomeAutomation/VideoSquirtGun.JPG
Watch this video of the Squirt Gun

Speech Recognition on the Kinect

One of the other features on the Kinect is the multi-array microphone with speech recognition. I played around with speech commands to control the squirt gun and to open the garage.

Watch this video of Simon Says Kinect

Android Garage Door Opener

I wanted to learn a little about Android development so I wrote a native Android app to call a REST web service (WCF) that talks to the netduino to open the garage.

Putting it all Together

The image below shows the communication between the components. NetduinoHomeAutomation/LogicalDan_005.jpg

The image below shows the devices that the Netduino controls. The fireplace project is in progress and I just started working on it.

NetduinoHomeAutomation/LogicalDan_006.jpg

NetduinoHomeAutomation/VideoAll.JPG
Watch this video to see how it all comes together

Ethernet Communication

The Ethernet communication with the netduino was the hardest part of the project.

C#
using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Microsoft.SPOT;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.Netduino;
using Microsoft.SPOT.Net.NetworkInformation;
using System.Threading;

namespace Netduino.Controller
{
    public delegate void MessageEventHandler(string Message);

    class EthernetCommunication
    {
        #region Private Variables
        private string _hostAddress = null;
        private int _port = 80;
        private string _netduinoStaticIPAddress = null;
        private string _subnetMask = null;
        private string _gatewayAddress = null;
        private Thread _listeningThread;
        private Socket _clientSocket = null;
        private static EthernetCommunication _ethernetCommunication;
        #endregion

        #region Constructors
        //This keeps other classes from creating an instance
        private EthernetCommunication()
        {
        }
        #endregion

        #region Public Properties
        public string HostAddress
        {
            set { _hostAddress = value; }
            get { return _hostAddress; }
        }
        public int Port
        {
            set { _port = value; }
            get { return _port; }
        }
        public string NetduinoStaticIPAddress
        {
            set 
            { 
                _netduinoStaticIPAddress = value;
                SetNetduinoStaticIPConfiguration();
            }
            get { return _netduinoStaticIPAddress; }
        }
        public string SubnetMask
        {
            set
            {
                _subnetMask = value;
                SetNetduinoStaticIPConfiguration();
            }
            get { return _subnetMask; }
        }
        public string GatewayAddress
        {
            set 
            {
                _gatewayAddress = value;
                SetNetduinoStaticIPConfiguration();
            }
            get { return _gatewayAddress; }
        }
        #endregion

        #region Events
        public static event MessageEventHandler EventHandlerMessageReceived;
        #endregion

        #region Public Methods
        
        private void StartListening()
        {
            _listeningThread = new Thread(new ThreadStart(ReceiveSocketsInListeningThreadAndHandleSocketExceptions));
            _listeningThread.Start();
        }

        private void InitializeConfiguration()
        {
            if (_netduinoStaticIPAddress == null)
                throw new Exception("The netduino Static IP Address nust be set!");

            if (_subnetMask == null)
                throw new Exception("The Subnet Mask must be set!");

            if (_gatewayAddress == null)
                throw new Exception("The Gateway address must be set.");

            SetNetduinoStaticIPConfiguration();
            NetworkInterface networkInterface = NetworkInterface.GetAllNetworkInterfaces()[0];

            if (_netduinoStaticIPAddress != networkInterface.IPAddress)
                throw new Exception("Problem setting the static IP.");

            if (_subnetMask != networkInterface.SubnetMask)
                throw new Exception("Problem setting the subnet mask.");

            if (_gatewayAddress != networkInterface.GatewayAddress)
                throw new Exception("Problem setting the gateway address.");
        }
        #endregion

        #region Public Static Methods
        public static EthernetCommunication GetInstance()
        {
            if (_ethernetCommunication == null)
            {
                _ethernetCommunication = new EthernetCommunication();
                _ethernetCommunication.HostAddress = Config.HostAddress;
                _ethernetCommunication.Port = Config.Port;
                _ethernetCommunication.NetduinoStaticIPAddress = Config.NetduinoStaticIPAddress;
                _ethernetCommunication.SubnetMask = Config.SubnetMask;
                _ethernetCommunication.GatewayAddress = Config.GatewayAddress;
                _ethernetCommunication.InitializeConfiguration();
                _ethernetCommunication.StartListening();
            }
            return _ethernetCommunication;
        }

        public static void SendMessage(string message)
        {
            GetInstance().SendEthernetMessage(message);
        }
        #endregion

        #region Private Methods
        private bool IsSocketConnected(Socket socket)
        {
            bool connectionNotClosedResetOrTerminated = !socket.Poll(1000, SelectMode.SelectRead);
            bool socketHasDataAvailableToRead = (socket.Available != 0);
            return (connectionNotClosedResetOrTerminated || socketHasDataAvailableToRead);
        }

        private void ReceiveSocketsInListeningThreadAndHandleSocketExceptions()
        {
            try
            {
                ReceiveSocketsInListeningThread();
            }
            catch (SocketException se)
            {
                Debug.Print("Socket Exception!  Probably WiFi or Ethernet connection not working?");
                Debug.Print(se.StackTrace);

                Debug.Print("Rebooting netduino to recover.");
                PowerState.RebootDevice(false);
            }
            catch (Exception ex)
            {
                Debug.Print("Non socket exception.");
                Debug.Print(ex.StackTrace);
            }
        }

        private void ReceiveSocketsInListeningThread()
        {
            string receiveMessage = "";
            bool exitProgram = false;

            using (System.Net.Sockets.Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
            {
                socket.Bind(new IPEndPoint(IPAddress.Any, _port));
                socket.Listen(10);

                while (!exitProgram)
                {
                    Debug.Print("Waiting for message from socket...");

                    // This call is "blocking" and will will wait for a socket.  
                    // The thread will wait here until a message is received
                    _clientSocket = socket.Accept();

                    Debug.Print("Message received!");

                    using (_clientSocket)
                    {
                        while (IsSocketConnected(_clientSocket))
                        {
                            int availablebytes = _clientSocket.Available;
                            byte[] buffer = new byte[availablebytes];
                            _clientSocket.Receive(buffer);
                            if (buffer.Length > 0)
                            {
                                receiveMessage = new string(Encoding.UTF8.GetChars(buffer));
                                RaiseMessageReceivedEvent(receiveMessage);
                                if (receiveMessage.ToUpper() == "EXIT")
                                {
                                    exitProgram = true;
                                }
                            }
                            
                        }
                    }
                }
            }
        }

        private void RaiseMessageReceivedEvent(string message)
        {
            // Event will be null if there are no subscribers
            if (EventHandlerMessageReceived != null)
            {
                EventHandlerMessageReceived(message);
            }
        }

        private void SetNetduinoStaticIPConfiguration()
        {
            //Exit if not all of the configuration properties are set
            if (_netduinoStaticIPAddress == null || _subnetMask == null || _gatewayAddress == null)
                return;

            NetworkInterface networkInterface = NetworkInterface.GetAllNetworkInterfaces()[0];

            bool _ipAddressAlreadySet = _netduinoStaticIPAddress == networkInterface.IPAddress;
            bool _subnetMaskAlreadySet = _subnetMask == networkInterface.SubnetMask;
            bool _gatewayAlreadySet = _gatewayAddress == networkInterface.GatewayAddress;

            if (_ipAddressAlreadySet && _subnetMaskAlreadySet && _gatewayAlreadySet)
                return;

            // Set our IP address to a new value
            // This will be saved in the config sector of the netduino and will survive reboots 
            networkInterface.EnableStaticIP(_netduinoStaticIPAddress, _subnetMask, _gatewayAddress);
        }

        private void SendEthernetMessage(string message)
        {
            if (_hostAddress != null && _port > 0)
            {
                using (System.Net.Sockets.Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
                {
                    IPHostEntry entry = Dns.GetHostEntry(_hostAddress);
                    IPAddress address = entry.AddressList[0];
                    IPEndPoint endpoint = new IPEndPoint(address, _port);

                    try
                    {
                        socket.Connect(endpoint);
                        socket.Send(Encoding.UTF8.GetBytes(message));
                        socket.Close();
                        Debug.Print(message);
                    }
                    catch (SocketException se)
                    {
                        Debug.Print("Socket Exception!  Probably no server or bad ip?");
                        Debug.Print(se.StackTrace);

                        Debug.Print("Rebooting netduino to recover.");
                        PowerState.RebootDevice(false);
                    }
                    catch (Exception ex)
                    {
                        Debug.Print("Non socket exception.");
                        Debug.Print(ex.StackTrace);
                    }
                }
            }
        }
        #endregion
    }
}

If you want to see an example of how to talk with a desktop application, download the source code and look at the Netduino.Desktop.Messenger project.

Servo Communication

The servos were fun to program. I wrote a servo class that you set the Angle and the Minimum and Maximum degrees. I added the Inverted property to invert the angle because I have an indoor version of the squirt gun that is mounted from the floor while the outdoor version is mounted upside down.

C#
using System;
using Microsoft.SPOT.Hardware;
using SecretLabs.NETMF.Hardware;

namespace Netduino.Controller
{
    public class Servo : IDisposable
    {
        #region Private Variables
        private PWM _servo;
        private bool _invertAngle = false;
        private int _degreeMin = Config.DegreeMinDefault;
        private int _degreeMax = Config.DegreeMaxDefault;
        private uint _durationMin = Config.DurationMinDefault;
        private uint _durationMax = Config.DurationMaxDefault;
        private uint _angle = Config.HomeDefaultAngle;
        private uint _period = Config.PeriodDefault;
        #endregion

        #region Constructors
        public Servo(Cpu.Pin pin)
        {
            _servo = new PWM(pin);
            _servo.SetDutyCycle(0);
        }
        #endregion

        #region Public Methods
        public void Dispose()
        {
            DisengageServo();
            _servo.Dispose();
        }

        /// <summary> 
        /// Disengage the servo.  
        /// The servo motor will stop trying to maintain an angle 
        ///  
        public void DisengageServo()
        {
            _servo.SetDutyCycle(0);
        }

        public void EngageServo()
        {
            SetPulse();
        }
        #endregion

        #region Private Methods
        private void SetPulse()
        {
            uint angle = _invertAngle ? 180 - _angle: _angle;
            uint duration = (angle) * (_durationMax - _durationMin) / 180 + _durationMin;
            _servo.SetPulse(period: _period, duration: duration);  
        }
        #endregion

        #region Public Properties
        public int Angle
        {
            set
            {
                if (value > _degreeMax)
                    value = _degreeMax;

                if (value < _degreeMin)
                    value = _degreeMin;

                if (value < 0)
                    value = 0;

                _angle = (uint)value;
                SetPulse();
            }
            get
            {
                return (int)_angle;
            }
        }

        public bool InvertAngle
        {
            set {_invertAngle = value;}
            get { return _invertAngle; }
        }

        public int DegreeMin
        {
            set {_degreeMin  = value;}
            get { return _degreeMin; }
        }

        public int DegreeMax
        {
            set { _degreeMax = value; }
            get { return _degreeMax; }
        }

        public uint durationMin
        {
            set { _durationMin = value; }
            get { return _durationMin; }
        }

        public uint durationMax
        {
            set { _durationMax = value; }
            get { return _durationMax; }
        }

        public uint period
        {
            set { _period = value; }
            get { return _period; }
        }
        #endregion
    }
}

Controlling the Garden

I wrote a library of time commands for the .NET Micro Framework. Note that the .net micro framework is very rich but does not have generics.

C#
using System;
using System.Threading;
using System.Collections;
using Microsoft.SPOT;

namespace Netduino.Controller
{
    public delegate void AlarmCallback();

    class AlarmData
    {
        public int Key { get; set; }
        public ExtendedTimer ExtendedTimer { get; set; }
        public bool RemoveAfterRun { get; set; }
        public AlarmCallback Callback { get; set; }
    }

    class Time
    {
        #region Private Variables
        private static Hashtable _alarmHashtable;
        private static int _key;
        #endregion

        #region Constructors
        //This keeps other classes from creating an instance
        private Time()
        {
        }
        #endregion

        #region Public Static Methods
        public static void SetTime(int year, int month, int day, int hour,int minute, int second, int millisecond )
        {
            DateTime presentTime = new DateTime( year, month, day, hour, minute, second, millisecond);
            Microsoft.SPOT.Hardware.Utility.SetLocalTime(presentTime);
        }

        public static void RunDaily(AlarmCallback alarmCallback, int hour, int minute, int second)
        {
            DateTime alarmTime = new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day, hour, minute, second, 0);
            
            //If we already missed today then tomorrow is the first day to run
            if(alarmTime<DateTime.Now)
            {
                alarmTime = alarmTime.AddDays(1);
            }

            TimeSpan dailyTimeSpan = new TimeSpan(24, 0, 0);
            CreateAlarm(alarmCallback, alarmTime, dailyTimeSpan, false);
        }

        public static void RunOnDelay(AlarmCallback alarmCallback, int runInMilliseconds)
        {
            DateTime alarmTime = DateTime.Now.AddMilliseconds(runInMilliseconds);
            CreateAlarm(alarmCallback, alarmTime, TimeSpan.Zero, true);
        }

        public static void RunRepetitively(AlarmCallback alarmCallback, int repeatMilliseconds)
        {
            DateTime alarmTime = DateTime.Now.AddMilliseconds(repeatMilliseconds);
            TimeSpan repeatTimeSpan = new TimeSpan(0, 0, 0, 0, repeatMilliseconds);
            CreateAlarm(alarmCallback, alarmTime, repeatTimeSpan, false);
        }
        #endregion

        #region Private Methods
        private static void CreateAlarm(AlarmCallback alarmCallback, DateTime alarmTime, TimeSpan timeSpan, bool removeAfterRun)
        {
            if (_alarmHashtable == null)
                _alarmHashtable = new Hashtable();
            
            _key=_key+1;

            AlarmData alarmData = new AlarmData();
            alarmData.Key = _key;
            alarmData.Callback = alarmCallback;
            alarmData.ExtendedTimer = new ExtendedTimer(OnExecuteAlarm, alarmData, alarmTime, timeSpan);
            alarmData.RemoveAfterRun = removeAfterRun;

            _alarmHashtable.Add(_key, alarmData);
        }

        private static void OnExecuteAlarm(object target)
        {
            AlarmData alarmData = (AlarmData)target;
            
            if (alarmData.RemoveAfterRun)
                _alarmHashtable.Remove(alarmData.Key);

            alarmData.Callback.Invoke();
        }
        #endregion
    }
}

The Main .NET Micro Framework Program

There is an event handler for the Ethernet Communication that runs when a message is received.

C#
EthernetCommunication.EventHandlerMessageReceived += new MessageEventHandler(OnMessageReceived);

The OnMessageReceived method parses the message and calls the methods to execute the commands. The code snippet below is only partial, but the full source is available for download in this article.

C#
private static void OnMessageReceived(string message)
{
    string[] parts = message.Split(' ');
    switch(parts[0].ToUpper())
    {
        case "M":
        case "MOVE":
            if (parts.Length != 3)
            {
                EthernetCommunication.SendMessage("The move command takes 3 arguments.");
                break;
            }
            int leftRightAngle = int.Parse(parts[1]);
            int upDownAngle = int.Parse(parts[2]);
            _squirtGun.MoveToPosition(leftRightAngle, upDownAngle);
            break;

        case "U":
        case "UP":
            int upDelta = parts.Length > 1 ? int.Parse(parts[1]) : 1;
            _squirtGun.UpDownAngle = _squirtGun.UpDownAngle + upDelta;
            break;

About the Author

Follow Dan on twitter: @LogicalDan

Dan graduated summa cum laude from North Carolina State University with dual degrees in Electrical Engineering and Computer Engineering. Dan attended NC State on full scholarship program with General Motors. After working with GM, Dan served as application development director for the largest Microsoft Business Solutions Partner in the Carolinas. During this time, Dan's team won two Microsoft Pinnacle awards. For the past 12 years, as Co-Founder and Chief Technology Officer of, Logical Advantage (www.logicaladvantage.com), a software consulting business, Dan has successfully architected and delivered web-based and mobile applications for many Fortune 500 companies. Dan focuses his energies on emerging technologies, and ensuring that all projects are architected to meet the client's current and future needs. Dan collaborates with his Chief Solutions Officer and other architects to create technical standards, including coding standards, tools, and platforms. He holds a leadership role in the local Microsoft Enterprise Developer's Guild and has been on the steering committee for over a dozen years.

Download the Source Code

Click this to download the source code for the netduino .net micro framework projects.

Continue Reading my Home Automation Series:

Click this link to read part 2 of my home automation projects: Using jQuery Mobile with MVC and Netduino for Home Automation

Click this link to read part 3 of my home automation projects: Home Automation with Microsoft Kinect Point Cloud and Speech Recognition

Click this link to read part 4 of my home automation projects: IoT for Home Automation

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Chief Technology Officer Logical Advantage
United States United States
Dan graduated summa cum laude from North Carolina State University with dual degrees in Electrical Engineering and Computer Engineering. Dan attended NC State on full scholarship program with General Motors. After working with GM, Dan served as application development director for the largest Microsoft Business Solutions Partner in the Carolinas. During this time, Dan's team won two Microsoft Pinnacle awards. For the past 10 years, as Co-Founder and Chief Technology Officer of, Logical Advantage (www.logicaladvantage.com), a software consulting business, Dan has successfully architected and delivered web-based and mobile applications for many Fortune 500 companies. Dan focuses his energies on emerging technologies, and ensuring that all projects are architected to meet the client's current and future needs. Dan collaborates with his Chief Solutions Officer and other architects to create technical standards, including coding standards, tools, and platforms. He holds a leadership role in the local Microsoft Enterprise Developer's Guild and has been on the steering committee for over a dozen years.

Comments and Discussions

 
GeneralRe: Something I don't see everyday - My vote is 5 Pin
Mpho aka Zazo15-Apr-12 20:26
Mpho aka Zazo15-Apr-12 20:26 
GeneralRe: Something I don't see everyday - My vote is 5 Pin
Dan Thyer16-Apr-12 3:24
Dan Thyer16-Apr-12 3:24 
GeneralRe: Something I don't see everyday - My vote is 5 Pin
Mpho aka Zazo16-Apr-12 20:33
Mpho aka Zazo16-Apr-12 20:33 
QuestionGood work Pin
Member 456543330-Mar-12 10:35
Member 456543330-Mar-12 10:35 
AnswerRe: Good work Pin
Dan Thyer30-Mar-12 10:43
Dan Thyer30-Mar-12 10:43 
QuestionVote of 5 indeed Pin
Martin Jimenez13-Mar-12 8:50
Martin Jimenez13-Mar-12 8:50 
AnswerRe: Vote of 5 indeed Pin
Dan Thyer30-Mar-12 10:48
Dan Thyer30-Mar-12 10:48 
QuestionI want to thank you, indeed. Pin
kartalyildirim3-Mar-12 6:36
kartalyildirim3-Mar-12 6:36 
I want to thank you, indeed.


modified 23-Mar-12 6:57am.

AnswerRe: I want to thank you, indeed. Pin
Dan Thyer3-Mar-12 6:40
Dan Thyer3-Mar-12 6:40 
GeneralMy vote of 5 Pin
RC_Sebastien_C19-Feb-12 4:57
RC_Sebastien_C19-Feb-12 4:57 
GeneralRe: My vote of 5 Pin
Dan Thyer20-Feb-12 4:36
Dan Thyer20-Feb-12 4:36 
GeneralMy vote of 5 Pin
David Catherman17-Feb-12 2:37
David Catherman17-Feb-12 2:37 
GeneralRe: My vote of 5 Pin
Dan Thyer17-Feb-12 9:27
Dan Thyer17-Feb-12 9:27 
QuestionGreat article! Pin
Member 865313216-Feb-12 12:06
Member 865313216-Feb-12 12:06 
AnswerRe: Great article! Pin
Dan Thyer16-Feb-12 12:50
Dan Thyer16-Feb-12 12:50 
GeneralMy vote of 5 Pin
kurtmerkle16-Feb-12 11:05
kurtmerkle16-Feb-12 11:05 
GeneralRe: My vote of 5 Pin
Dan Thyer16-Feb-12 12:49
Dan Thyer16-Feb-12 12:49 
Questionsome doubts Pin
kamaloo9-Feb-12 6:35
kamaloo9-Feb-12 6:35 
AnswerRe: some doubts Pin
Dan Thyer9-Feb-12 7:26
Dan Thyer9-Feb-12 7:26 
GeneralMy vote of 5 Pin
Ștefan-Mihai MOGA8-Feb-12 6:35
professionalȘtefan-Mihai MOGA8-Feb-12 6:35 
GeneralRe: My vote of 5 Pin
Dan Thyer9-Feb-12 7:02
Dan Thyer9-Feb-12 7:02 
GeneralMy vote of 5 Pin
Md. Marufuzzaman7-Feb-12 19:41
professionalMd. Marufuzzaman7-Feb-12 19:41 
GeneralRe: My vote of 5 Pin
Dan Thyer9-Feb-12 7:02
Dan Thyer9-Feb-12 7:02 
GeneralMy vote of 5 Pin
Jαved6-Feb-12 23:27
professionalJαved6-Feb-12 23:27 
GeneralRe: My vote of 5 Pin
Dan Thyer7-Feb-12 7:43
Dan Thyer7-Feb-12 7:43 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.