Click here to Skip to main content
14,872,052 members
Articles
Article
Posted 2 Aug 2016

Tagged as

(untagged)

Stats

58.9K views
147 downloads
15 bookmarked

Raspberry Pi - Part 3 IoT Hello World

Rate me:
Please Sign up or sign in to vote.
4.93/5 (8 votes)
9 Aug 2016CPOL22 min read
An IoT end-to-end Hello World tutorial using Node.js, Express, socket.io, and associated middleware and the ESP8266 WiFi adapter board.

Introduction

In this, the third in the Raspberry Pi series of articles we will be building on previous articles to develop a simple solution for what I would call an Internet of Things Hello World project. In my previous articles we discussed intially setting up the Raspberry Pi and in the second article using Mosquitto as a message broker. In this article we will be introducing serveral new software, middleware and firmware packages as well as a new hardware device the ESP8266-12e WiFi adapter. If you haven't read my previous articles I suggest you read them before continuing.

The image below illustrates the very crude web page, served by our Web Server that we'll be creating in this project. It will wirelessly control and display the status of an LED connected to gpio ports on the ESP8266-12e. Although this is a very basic project it contains all the components that would be needed for a more sophisticated application.

1116198/pi3-wiring-0.png

I've used a Raspberry Pi II throughout this series but just for the heck of it I took the card out of the RPi II and inserted it into my PiZero. I initially had a problem when the PiZero tried to boot but the problem turned out to be that I had a thumb drive mounted on the RPi II and on the PiZero I hadn't plugged it in, once I did plug it in it worked great. Bottom line both system's hardware must be configured exactly the same for it to work. I guess I could have unmounted the thumb drive and it would have worked, but I didn't try it as I use the drive as my development environment.

Table of Contents

We will be covering a lot of ground in this article so I have decided to break this tutorial up into two distinct areas; the first will be an overview of the ESP8266-12e and eLua language, a brief overview of the mqtt messaging and Web Server software and finally go into detail in section 3.0 concerning the software for the project itself.

1.0 ESP8266-12e WiFi addapter board

The ESP8266 incorporates a 32-bit RISC CPU based on the Tensilica Xtensa LX106 SoC running at 80MHz. Manufactured by the Shanghai based Espressif company it is a low cost WiFi chip with full TCP/IP stack and Microcontroller capability.

ESP8266 Specifications
Description Value
Voltage 3.3V NOT 5V
Current consumption 10uA - 170mA
Maximum drive capabilites 12mA
Flash memory attachable 16MB max (512K normal)
Processor Tensilica L106 32bit
Processor Speed 80MHz-160MHz
RAM 32K + 80K
GPIOs 17 (multiplexed)
Analog to Digital 1 input 10bit resolution
802.11 support b/g/n/d/e/i/k/r
Maximum concurrent TCP connections 5

There have been many versions of this board manufactured with various capabilities ranging from just a couple of gpio ports to a full blown suite of functions that you would normally find on a microcontroller such as the Arduino. The Operating System is event driven and not multitasking so care should be taken when using blocking code such as tmr.delay. The device as shipped from the factory supports the old style AT commands that you may, or may not be old enough to rememeber from the days of the Hayes modem. While these commands may be useful for a small project I don't think I would use it in a production environment. Instead the good people at NodeMCU have developed an open source interpreted language based on eLua that is more than capable for any project you have in mind. They are also in the process of creating a MicroPython version but as of this writing I don't think it's ready for comsumption yet.

1116198/pi3-esp8266-1.png

The board we will be using for this project is the ESP8266-12e NodeMCU DevKit as shown in the image above on the far right. The board integrates GPIO, PWM, I2C, ADC, 1-wire and SPI as well as UART via USB or through pins on board. The version I purchased has the CH340 chip instead of the official FTDI chip so you will need to check the version you have and install the appropriate windows drivers, if you are indeed on a windows machine. Fortunately there is a lot of documentation on the web about this board and a large community such as Everything nodeMCU that are willing to share there knowledge. I also ran across a free book Kolban's book on the ESP8266 that is a excellent reference. Below is the pinout for the device and as you can see the pins are multiplexed meaning they may be configured for multiple purposes.

1116198/pi3-esp8266-0.png

1.1 NodeMCU Flasher

In order to utilize the NodeMCU firmware it must be flashed onto the device and for this we need to download the ESP8266Flasher executible. The flasher application is fairly straight forward to use and there is plenty of documentation on site on it's usage. We get the firmware to flash onto the device from nodemcu-build.com, where you can pick what functionality you require for the device then have them build it for you. When they are done you will be emailed a link where the image can be downloaded. When flashing the device pin gpio0/D3 must be held low. On the ESP8266-1 this was an issue but on the ESP8266-12e this was not an issue as I believe they have the hardware on board to sense when it is being flashed. At any rate a heads up, if the flash isn't working this may be the reason.

1116198/pi3-nodemcu-0.png

1.2 ESPlorer

Once we have the device flashed with the latest NodeMCU firmware we need some way of programming it. I highly recommend the development IDE ESPlorer by Victor Brutskiy. It is a free application, written in Java that supports both the AT command and LUA based languages. It also has the option to program in MicroPython but I didn't try it so I have no idea if it works or if the language is available yet.

ESPlorer supported platforms;
	Windows(x86, x86-64)
	Linux(x86, x86-64, ARM Soft & hard float)
	Solaris(x86, x86-64)
	MacOS X(x86, x86-64, PPC, PPC64)

The ESPlorer IDE is a powerful tool but I only use a subset of the functionality that it provides. Instead of describing all the details of the IDE we will go through the process of creating, saving to the device as a file and running.

1116198/pi3-esplorer-0.png

The device keeps an area in non-volatile memory set up as a flat file system where files to be run may be stored. To format the file system on the device click the 'Format' button, caution this will clear the file area erasing all files that currently reside there. To get statistics on the file system click the 'FS Info' button, this will tell you how many files you have saved and there names as well as the total file area size and how much is in use and free. The last button grouped with the others is the 'Reload' button that displays/refreshes a list of buttons with file names below the reload button itself that when clicked will execute that file.

When the device boots up it looks for a file locally named init.lua and if found automatically runs it. Therefore during the development cycle I would recommend naming the main file something other than init.lua, such as test.lua, otherwise you will have to reflash the device.

I won't describe how to use the application or all the functionality available but instead will describe the important pieces throughout the tutorial to get done what we need to do and if you would like more information about the application or how to use it I will leave it to you to google, there has been a lot written about this application so you shouldn't have any problems.

1.2.1 Simple Blink application

To showcase some of the functionality of the EPS8266 device and how to use the IDE we'll create a simple application to blink an LED on gpio0/D3 on/off a set number of times. Additionally each time through the loop the current cnt will be printed to the terminal.

Before we can start coding we need to assemble the very simple circuit shown in the schematic below consisting of a current limiting resistor and an LED from port D3 to ground. The value of the resistor required will depend on the LED that you select and can be calculated from values that are obtained from the LED's datasheet. You will need to know the forward voltage drop (Vf) and the forward current (i) of the ODED and from this the resistance can be calculated by R = (Vs - Vf) / i where Vs is the supply voltage. As an example if we have an LED with Vf = 1.8V and i = 30mA respectively with a supply voltage of 3.3V then; R = (3.3V - 1.8V) / .030 = 50ohm. I used a 330ohm resistor in my example which worked fine and lights the LED but it is just a tad dimmer than what it would have been had I used a resistor of the value calculated.

1116198/pi3-esplorer-0.png

Now that we have things wired up we can concentrate on the code. As can be seen in the code snippet that follows the NodeMCU eLua language is very Python-ish syntactically and even if you don't know Python should be fairly easy to understand.

pin = 3        -- gpio0/D3
target = 5     -- Our target count
 
local function blinky_func(target_cnt)
  local cnt = 0
   while cnt < target_cnt do
        -- gpio.write(pin, state)
        --    where state = gpio.FLOAT, gpio.HIGH or gpio.LOW
        gpio.write(pin, gpio.HIGH)
        tmr.delay(1000000)   -- wait 1,000,000 us = 1 second CAUTION Blocking
        gpio.write(pin, gpio.LOW)
        tmr.delay(1000000)   -- wait 1,000,000 us = 1 second CAUTION Blocking
        cnt = cnt + 1
        print(cnt)
    end
end
 
-- gpio.mode(pin, mode[, gpio.PULLUP])
--      where mode = gpio.INPUT, gpio.OUTPUT
gpio.mode(pin, gpio.OUTPUT)
 
-- Call the blinky_func function
blinky_func(target)  

There is a lot more functionality available in the eLUA language (See NodeMCU firmware documentation ) and we will go into greater detail when we get to the actual IoT Hello World application. Once the code has been entered into the editor window there are a number of things we can do from here;

  1. control+s to save the file locally and to the devices file system and run it.
  2. Save to ESP - saves the file to the device and runs it.
  3. Send to ESP - downloads it to the device and runs it.
  4. Once it has been saved to the device you may click on the Reload button and a list of files that have been saved on the device will appear, you can then click on the file you wish to run.
  5. Lastly if the file has been saved to the device you can send the 'dofile("blinky_func.lua")' command to the device by entering the command in the bottom right textbox and clicking on the Send button just to the right of the textbox.

2.0 RaspberryPi + Node.js

We are going to put the ESP8266 WiFi adapter to the side for now and concentrating on the Pi side for a bite. We're going to learn the basics of Node.js Express and the middleware that we will be using later in our IoT Hello project.

Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine. While it is possible to use Node.js along it will be a lot easier using Express. Express itself is very limited and is essentially a series of middleware function calls. Middleware definition;

Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware function in the application's request-response cycle. The next middleware function is commonly denoted by a variable name next.

2.1 Install & Configure Node and dependencies

We'll start the installation by getting the core applications Node.js and the Node Package Manager (NOM). I always do the update/upgrades to get machine up to date before doing any installations, it's a good practice to get into.

sudo apt-get update
sudo apt-get upgrade
 
sudo apt-get install npm
sudo apt-get install node

Because we'll be using Express we need to install some middleware using the package manager. There are two ways to configure the development environment for developing Node.js applications; the first method involves putting the dependency files in an area on disk and when a project is created, links are used to access the required modules in the global area. Secondly the dependency files are downloaded into the local directory and a global area may still be used but not necessary. I have my environment set up using the first method of linking to a global area and I'll show the steps necessary to do it both ways.

Installing the application specific modules is next and we have the ability to control where the modules reside by using command line arguments when invoking the install commandls;

sudo npm [--save] [--global | -g] install module-name
	
	where the optional auguments;
		-save = Installs a copy locally as well as global area
		-global = Only create a copy in the global area.

If no arguments are supplied the files are copied only into the working directory. If you previously downloaded the middleware files into the global area or are starting a new project and want to reference them use the link option.

Below is a list of the middleware modules we will be using in this project.

  • ejs - Simple templating language that let's you generate HTML markup with plain JavaScript.
  • body-parser - Parses incoming request bodies.
  • mqtt - Publish/Subscribe messaging protocol.
  • morgan - HTTP request logger.
  • pm2 - Runs the application in the background, provides load balanving and keeps the process alive.

This series of commands establishes links to the modules we want to be able to access in the global area this action has the side effect of creating the node_modules subdirectory.

sudo npm link express
sudo npm link ejs
sudo npm link body-parser
sudo npm link mqtt
sudo npm link morgan

When we want to install the modules locally we use the install command line argument and if we want to also save a copy globally add the --save arguemnt.

sudo npm install express
sudo npm install ejs
sudo npm install body-parser
sudo npm install mqtt
sudo npm install morgan

2.2 Create a Project

We may have gotten a little ahead of ourselves, before we can download module(s) to our working copy don't we need a working copy?

Creating the application environment or working copy is something that will become second nature if you are to do a lot of Node.js development. It's fairly easy to do just takes a few steps and the rest is automated. Basically you create a directory, change it's security permissions and invoke npm init to allow Node to do the setup. The example shown below creates the project 'home-automation' subdirectory and initialize it.

sudo mkdir home-automation
sudo chmode 777 home-automation
cd home-autmation
sudo npm init

When npm init is invoked the application prompts for a series of questions and for the most part you can just accept the defaults, but remember the name you used for the entry point during the init, you will need to create this file in the root directory. A package.json file is created by the init process and the entry point name you specified will be used by Visual Studio Code when running or degugging the application or when starting the application using the sudo npm start command. I don't use the npm start method as it takes extra time to set things up. If I'm developing on the Pi I won't be using Visual Studio Code so I invoke directly using sudo node home-automation.js.

1116198/pi3-nodeinstall-0.png

Notice that I shortened the name to 'home' and the entry point to app.js. I chaged the name just because I wanted to and to show that they don't need to be the same and I changed the entry point because I don't want this to be confused with file name index that has other extensions such as; html, asp, etc..

There is one other way in which to get the dependency modules you need and this is what makes the package manager cool. You can work backwards by supplying a dependency group, such as the example given below in the package.json file;

"dependencies": {
	"body-parser": "1.15.2",
	"express": "4.13.4",
	"ejs": "2.4.2",
	"jade": "1.11.0",
	"mqtt": "1.12.0"
}

then issue the command:

sudo npm install

npm will go to the web and pull down the modules it needs and put them in the local node_modules direcory. I really like this system because not only does it allows the user that wants to share a project to include the package.json that contains the dependency group but the end user can also get the correct version of the module that is available.

2.3 Running the project

During the development phase I started the application in the foreground so that I could stop or restart it quickly using ctrl-c.

sudo node app.js

Once the project is complete and it's time for it to go into production you would then use the pm2 utility to start the application in the background as illustrated in the image that follows.

1116198/pi3-mosquitto-0.png

In the image above you will notice that I started mosquitto before the node application, it must be running to make an mqtt connection. Refer to my previous article concerning setting up the message broker, if you are unfamiliar with mosquitto or configuring it as a message broker.

3.0 IoT Hello World project (The Code)

Now it's time to put all the pieces together and roll it into the IoT Hello World project. Up to this point we've been using mqtt as our messaging protocol, it is very powerful and we will continue to use it but we lack one key piece of functionality and that is the ability to push data to the client. We'll be using the socket.io middleware to accomplish this.

3.1 Mosquitto

We've already gone over the mosquitto stuff in my previous article so we won't beat a dead horse. In the /etc/mosquitto/mosquitto.conf file you will need to modify the contents and at least be sure these fields are set correctly. Refer to section 2.3 when you are ready to run mosquitto.

# Change this value to your Raspberry Pi's port and IP address
listener 1883 192.168.254.20
 
# When this is set to true no security is used 
allow_anonymous true

3.2 Node.js

The app.js javascript file is the entry point for our Node.js web server application. In the snippet, from app.js that follows we configure our callback methods for our mqtt broker messaging. The first callback is triggered when a user connects and they are immediately subscribed to the 'node/1/state/' topic. The second callback is triggered when a 'state' message is recieved from the mqtt client and passed on to the socket.io middleware's emit method that pushes the 'state' message out to connected clients.

// ==================> mqtt Handlers <====================
// When a client connects we subscribe to mqtt state msg.
client.on('connect', function(){
    console.log('Connecting...');
    client.subscribe('node/1/state/');
})
 
// Handler to receive the state msgs that we subscribed to above, from the ESP device.
client.on('message', function(topic, msg){
    console.log(String('{ ' + topic + ': ' + String(msg) + ' }'));
	// Incomming msg will either me a 0 or a 1 and we push the current state
	//	to the client to update their state status.
    if (msg == 1) {
        io.emit('state', 'ON');
    }
    else
    {
        io.emit('state', 'OFF');
    }
});

The portion of the client code in the snippet below is the JavaScript code that is used in our bi-direction or push/pull protocol using socket.io. The first call back defines the action taken when the client connects to the server, it creates another callback that receives the 'state' message sent by the server and updates the '#testy' field. Yeah I'm bad at naming stuff. The lower part in the jquery ready area are the Button event handlers that just pass the 'cmd' message on to the server.

<script type='text/javascript'>
    var socket = io.connect('192.168.254.20:3000');
    <!-- On connection we set a handler for the 'state' msg -->
    socket.on('connect', function() {
        socket.on('state', function(msg) {
            console.log(String('on.state: ' + String(msg)));
            $('#testy').text(String(msg));
        });
    });
    $(document).ready(function() {
        <!-- This keeps the form from submitting! -->
        $('form').submit(function(){ return false; });
        <!--
			Click handlers for the two buttons.  Send the appropriate
			'cmd' message to the server.
		-->
        $('#btnOnId').click(
            function($e) {
                socket.emit('cmd', 'ON');
                }
        ),
        $('#btnOffId').click(
            function($e) {
                socket.emit('cmd', 'OFF');
            }
        )
    });
</script>

3.3 ESP8266-12e eLua Code

When I started looking for example code for this project I had a stroke of good luck and found pretty much a piece of drop in code from a gentleman at foobarflier.io...thanks man if you're reading this. Simple connected object with NodeMCU and MQTT . I modified the code to suite my needs and commented the crap out of it but the firmware was well written so the mods were minimal and went very well.

Before powering up the ESP8266 we need to wire the LED and the feedback loop that we'll use for our project. Remember that the pin numbers are what the firmware uses to reference the ports not the gpio numbers.

1116198/pi3-esplorer-0.png

Once you have downloaded and unzipped the files fire up ESPlorer and load the files in the Raspi-Part3\code\lua directory into the editor;

  • config.lua - Contains the config settings for the project. You will need to edit this file and replace;
    • module.SSID - Replace with your SSID informationion
    • module.HOST - Host IP address
    • module.PORT - Port number you'll be using
    • module.CMD_PIN - ESP8266 gpio pin to use to control the LED.
    • module.STATE_PIN - ESP8266 gpio pin to use for the LED sate.
  • setup.lua - Firmware that establishes network connection to
  • application.lua - Establishes mqtt connection and contains callbacks.
  • test.lua - Entry point for the application.

Now download the files into the device and run the test.lua module, you should see the connection messages if everything is working as it should. Remember to leave the test.lua file name as it is until you are sure that the code is as you want it, then change the name to init.lua to have it auto start.

3.4 Running the IoT Hello project

Since we've covered most of this material in previous sections I have run through the IoT Hello project fairly quick. The only thing new was the socket.io stuff and it didn't prove to be a significant change to waht we had before. But for the benefit of completeness I will run through the steps that need to be performed to get this project up and running. If you have any problems please read the next section on Trouble Shooting.

  1. On the Pi install Mosquitto, NPM and middleware, if you haven't done so already.
  2. Unzip the download file into a working directory.
  3. Navigate to the code->node->IoT-Hello directory, transfer it and it's contents to a working directory on the Pi.
  4. On the Pi edit /etc/mosquitto/mosquitto.conf and replace the network information.
  5. Start mosquitto
  6. From a terminal on the Pi navigate to the IoT-Hello directory and invoke the sudo npm install command to install the required middleware.
  7. Edit app.js you will need to modify the g_broker variable with your network information.
  8. Start app.js
  9. Set up the LED and associated hardware on you breadboard.
  10. On the PC fire up the ESPlorer application and connect to the ESP8266-12e.
  11. Load the files in the code->lua directory.
  12. In the config.lua modify the SSID, HOST information and other as needed.
  13. Download the files to the ESP device and start the test.lua file.
  14. Fire up your favorite browser, navigate to the address and port you specified in the app.js file and if everything worked you should see the page displayed at the beginning of this tutorial and be able to click on the buttons and control the LED and the state should update.

I realize there are a lot of steps involved in getting this whole thing working and is pretty involved for just a simple application but I believe the whole process is fairly straight forward. On the plus side most, if not all of these steps would be necessary for a more sophisticated application.

3.5 Trouble Shooting

So you followed the instructions, to the letter and it doesn't seem to work. During the course of developing this project I formulated some debugging techniques that I'll pass on. One piece of advice, the mqtt_spy utility is your best friend!

First let's be sure that Mosquitto is running correctly. I found the best way to do this is to run it interactively,

sudo mosquitto -c /etc/mosquitto/mosquitto.conf

if it runs without giving an error message fire up mqtt_spy and publish/subscribe to any topic, "test" for instance and publish a test message and see if it shows up in the subscription pane. If it does then the mqtt stuff is working. If you got errors on startup or the test message didn't make it through check your settings in /etc/mosquitto/mosquitto.conf. Once Mosquitto is running properly you can move on but don't move on unless it is.

The next step is to get the ESP8266 LED and loopback wired up and the firmware running. If it is running correctly you should see the connection information displayed in the ESPlorer output pane. If the firmware gives errors or it seems to hang check the settings in config.lua file. Once you do get a connection established you can fire up mqtt_spy and publish an 'ON' or 'OFF' message under the 'node/1/cmd/' topic to control the LED. Likewise you can subscribe to the 'node/1/state/' topic to recieve state updates.

If you got all that working and you can't access the web page or it doesn't do anything when you click either of the buttons check the settings in the app.js file, that's really the only thing that can functionally go wrong. If you didn't set the project up correctly that's another story and in that case you need to go back and re-read that section of the tutorial.

And last but certainly not least if everything works correctly my name is Mike Hankey and if not my name is Elmer Fudd. Happy coding!

4.0 Summary

I did a lot of research before starting this project because there were so many unknowns. I didn't know LUA, or Node.js, although I had been looking at doing something with either Node.js or SignalR, I quickly ruled out SignalR as I found it confusing and didn't really do what I thought I wanted to do with it so decided to concentrate on Node.js and I'm glad I did. Quite frankly my biggest hold up was that I thought integrating the mqtt messaging with a web page would be difficult but after working with Node.js for a while I found it played very nicely with mqtt. And although Node.js is a great package I really like working with Express on top of Node.js and the fact that you can load only the middleware you need and there seems to be modules to do about whatever a person could want. Although the names are not very disciptive; grunt, busboy, etc..

I live above an outfitters store in a very small town in North central Florida and during the summer months business gets very slow so we have to tighten our belts and one of the luxuries we had to give up this summer was our internet service. But in town we have a brand new public library that has WiFi so I take my Pi up there and I think the old ladies get a kick out of me bringing my new fangled toy to the library, they make a big fuss and clear a table for me where I can work. I've found the Pi to be a very portable and affordable solution to my problem.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Mike Hankey
Retired
United States United States
Currently enjoying retirement and working on projects without pressure, deadlines or any kind of management.

Comments and Discussions

 
SuggestionLED current. Pin
SteveHolle9-Aug-16 7:24
MemberSteveHolle9-Aug-16 7:24 
GeneralRe: LED current. Pin
Mike Hankey9-Aug-16 7:47
professionalMike Hankey9-Aug-16 7:47 
QuestionLinks to Part 1 & 2 Hijacked Pin
Member 104156113-Aug-16 8:42
MemberMember 104156113-Aug-16 8:42 
AnswerRe: Links to Part 1 & 2 Hijacked Pin
Mike Hankey3-Aug-16 12:23
professionalMike Hankey3-Aug-16 12:23 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.