Click here to Skip to main content
13,799,972 members
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

5.7K views
6 bookmarked
Posted 23 May 2018
Licenced CPOL

Setting Up a Time Series for Sensor Data Using Amazon Web Services (AWS) Greengrass and the UP Squared Board

, 23 May 2018
This article explores the method for setting up a time series for sensor data using the UP Squared board and Grove shield, the Amazon Web Services (AWS) Greengrass, and Plotly.

Editorial Note

This article is in the Product Showcase section for our sponsors at CodeProject. These articles are intended to provide you with information on products and services that we consider useful and of value to developers.

Intro

This article explores the method for setting up a time series for sensor data using the UP Squared* board and Grove shield, the Amazon Web Services (AWS)* Greengrass, and Plotly*. First, we will collect the data from Grove’s UV sensor then we will create a time series using Plotly. Finally, we will publish the time series URL to Greengrass’s IoT Cloud using the AWS Lambda function.

Learn more about the AWS* Greengrass

Learn more about the UP Squared board

Learn more about Plotly

Prerequisites

UP Squared board:

AWS Greengrass:

Plotly:

AWS Greengrass

To install AWS Greengrass, follow these instructions

Check that you have installed all the needed dependencies:

sudo apt update
git clone https://github.com/aws-samples/aws-greengrass-samples.git
cd aws-greengrass-samples
cd greengrass-dependency-checker-GGCv1.3.0
sudo ./check_ggc_dependencies
Code 1. Commands to Check AWS Dependencies

On UP Squared board, start Greengrass:

cd <path-to-greengrass-folder>/ggc/core
sudo ./greengrassd start
Code 2. Commands to Start AWS Greengrass

Time Series

On UP Squared board, install Plotly and its dependencies:

sudo pip install pandas
sudo pip install flask
sudo apt-get install sqlite3 qlite3-dev
sudo pip install plotly
Code 3. Commands to Plotly with Dependencies

To update Plotly:

sudo pip install plotly --upgrade
Code 4. Commands to Update Plotly

Go to your home directory and create a new directory:

cd ~
mkdir .plotly
Code 5. Commands to Create a Directory

Get the API key and your login info from the Plotly account page. Create .credentials file in the .plotly directory:

{
    "username": "your-username",
    "stream_ids": [],
    "api_key": "your-api-key"
}
Code 6. .credentials file

You will need your username and API key information in the next section to authenticate with Plotly.

Grove Sensors

To interface with Grove sensors, install MRAA and UPM libraries:

sudo add-apt-repository ppa:mraa/mraa
sudo apt-get update
sudo apt-get install libmraa1 libmraa-dev mraa-tools python-mraa python3-mraa
sudo apt-get install libupm-dev libupm-java python-upm python3-upm node-upm upm-example
Code 7. Commands to Install Grove Dependencies

UV Sensor

This section shows how to collect data with the Grove UV sensor and save the sensor data in a CSV. The second script reads the CSV file and creates a time series using Plotly and Pandas Python* packages. The URL for time series is saved in a file that is used by the AWS Lambda function in the next section. Code used in this section was modified from the source code, which can be found here. Here’s the modified code we will use to retrieve and save the sensor data:

from __future__ import print_function
import time, sys, signal, atexit
​from datetime import datetime
from upm import pyupm_si114x as upmSi114x
import mraa

def main():
    # Interface with sensors through the Grove shield
    mraa.addSubplatform(mraa.GROVEPI, "0")

    # Instantiate a SI114x UV Sensor on I2C bus 0
    myUVSensor = upmSi114x.SI114X(0)

    ## Exit handlers ##
    # This stops python from printing a stacktrace when you hit control-C
    def SIGINTHandler(signum, frame):
        raise SystemExit

    # This function lets you run code on exit,
    # including functions from myUVSensor
    def exitHandler():
        print("Exiting")
        sys.exit(0)

    # Register exit handlers
    atexit.register(exitHandler)
    signal.signal(signal.SIGINT, SIGINTHandler)

    # First initialize it
    myUVSensor.initialize()

    print("UV Index Scale:")
    print("---------------")
    print("11+        Extreme")
    print("8-10       Very High")
    print("6-7        High")
    print("3-5        Moderate")
    print("0-2        Low\n")

    # update every second and print the currently measured UV Index
    while (1):
        # update current value(s)
        myUVSensor.update()
        uv = myUVSensor.getUVIndex()
        # print detected value
        print("UV Index:", uv)
        dt = datetime.now()

        with open("uv_data.csv", "a+") as uv_data_file:
            uv_data_file.write(str(dt) + ',' + str(uv) +',\n')
        time.sleep(10)

if __name__ == '__main__':
    main()
Code 8. uv_sensor.py, Python Code to Get and Record UV Sensor Data

The Python code gets the data from Grove UV sensor and writes it to the uv_data.csv, with a timestamp.

Save the code as uv_sensor.py on the UP Squared board. To collect the UV sensor data, run the code:

sudo python uv_sensor.py 
Code 9. Command to Run Python Code

After you’ve collected enough data, press Ctrl+C to stop running code.

Save create_plot.py:

import plotly.plotly as py
import plotly.graph_objs as go
import pandas as pd
from datetime import datetime


# Plotly account authentication
py.sign_in('plotly-username', 'plotly-api-key')

# Reading CSV file using Pandas package
df = pd.read_csv("uv_data.csv", header=None, parse_dates=True, infer_datetime_format=True, usecols=[0,1])

# Creating a time series with Plotly package
data = [go.Scatter(
          x=df[0],
          y=df[1])]
url = py.plot(data)

# Saving URL to a file
with open("url_file", "a+") as url_file:
    url_file.write(str(url))
Code 10. create_plot.py, Python Code to Create Time Series

Replace plotly-username and plotly-api-key with your Plotly credentials.

This code reads the UV sensor data stored in CSV file and creates a time series that is displayed online. The URL is written to a file, so AWS Lambda can use it later in the tutorial.

Run the code:

sudo python create_plot.py
Code 11. Command to Run Python Code

AWS Lambda

This section shows you how to prepare for and create the AWS Lambda function. This function will read the time series URL from a file and then publish it via MQTT client to the Greengrass’s IoT Cloud. At the end, we will see the MQTT messages received and the time series with UV sensor data.

Go to AWS console and then to the AWS IoT page and select Software from the bottom left. Download the AWS Greengrass Core SDK by clicking on Configure Download. Choose Python* 2.7 and click Download Greengrass Core SDK. After the package has loaded, untar it:

tar –xzvf greengrass-core-python-sdk-1.0.0.tar.gz
Code 12. Command to Untar a Package

Go to the HelloWorld folder:

cd aws_greengrass_core_sdk/examples/HelloWorld
Code 13. Command to Go to the HelloWorld Folder

Unzip the zip file:

unzip greengrassHelloWorld.zip
Code 14. Command to Unzip A Package

Copy url_file to the same folder:

cp <path-to-url-file>/url_file .
Code 15. Command to Copy a File

Copy time_series.py and save it in the same folder:

import greengrasssdk
import platform
from threading import Timer
import time


# Creating a greengrass core sdk client
client = greengrasssdk.client('iot-data')

# Retrieving platform information to send from Greengrass Core
my_platform = platform.platform()


def time_series_run():
    fopen = open("url_file", "r")
    url = fopen.read()
    if not my_platform:
        client.publish(topic='ts/uv', payload='View the time series: {} Sent from Greengrass Core.'.format(url))
    else:
        client.publish(topic='ts/uv', payload='View the time series: {} Sent from Greengrass Core running on platform'.format(url))

    # Asynchronously schedule this function to be run again in 10 seconds    
    Timer(10, time_series_run).start()


# Start executing the function above
time_series_run()

def function_handler(event, context):
    return
Code 16. time_series.py, Python Code to Publish Time Series URL to AWS Greengrass

Create a zip file, timeseries.zip, which will be later uploaded to the AWS Lambda function:

zip –r uv_timeseries.zip greengrass_common/ greengrass_ipc_python_sdk/ greengrasssdk/ time_series.py
Code 17. Command to Create a Zip File

Go to AWS console, click Services on top left, put Lambda in search bar and click on it. The Lambda Management Console will open.

AWS Lambda Functions View

Figure 1. AWS Lambda Functions View

Click Create function.

If not selected, select Author from scratch and fill out needed fields:

AWS Lambda Create Function View

Figure 2. AWS Lambda Create Function View

Click Create function.

Upload time_series.zip. Change handler name to time_series.function_handler. Click Save:

Creating AWS Lambda Function View

Figure 3. Creating AWS Lambda Function View

Click on Actions, select Create new version and call it the first version in description:

Publishing New Version of AWS Lambda Function

Figure 4. Publishing New Version of AWS Lambda Function

Click Publish.

Go to IoT Core/AWS IoT console. Choose Greengrass from leftside menu, select Groups underneath it, and select your group from main window:

AWS Greengrass Groups View

Figure 5. AWS Greengrass Groups View

Select Lambda from the leftside menu. Click Add Lambda on right top corner of Lambdas screen:

AWS Greengrass Group View

Figure 6. AWS Greengrass Group View

Select Use Existing Lambda:

Adding AWS Lambda Function for the Greengrass Group

Figure 7. Adding AWS Lambda Function for the Greengrass Group

Select time_series from the menu and click Next:

Using Existing AWS Lambda

Figure 8. Using Existing AWS Lambda

Choose Version 1 and click Finish:

selecting Lambda version

Figure 9. Selecting Lambda Version

Click on dotted area and select Edit Configuration:

select dotted area and select Edit Configuration

Figure 10. Lambda Functions Within Greengrass Group View

Change Timeout to 25 seconds and choose Lambda lifecycle to be a long-lived function:

Editing Lambda function view

Figure 11. Editing Lambda Function View

Click Update on the bottom of the page.

Click the little grey back button, select Subscriptions.

Click Add Subscription or Add your first Subscription:

click add subscription

Figure 12. Adding Subscription View

For the source, choose from Lambdas tab, select time_series. For the target, select IoT Cloud:

edit subscription view

Figure 13. Editing Subscription View

Click Next. Add ts/uv for the topic:

click next and add ts/uv for the topic

Figure 14. Editing Topic for Subscription View

Click Finish.

On the group header, click Actions, select Deploy and wait until it is successfully completed:

subscriptions view

Figure 15. Subscriptions View

Go to the AWS IoT console. Select Test from the leftside menu. Type ts/uv in the topic field, change MQTT payload display to display it as strings, and click Subscribe to topic:

MQTT client view

Figure 16. MQTT Client View

After some time, messages should display on the bottom of the screen:

MQTT messages view

Figure 17. MQTT Messages View

Copy the URL and paste it in browser. The time series of UV sensor data is shown below:

UV sensor data time series

Figure 18. UV Sensor Data Time Series

About the author

Rozaliya Everstova is a software engineer at Intel in the Software and Services Group working on scale enabling projects for Internet of Things.

More on UP Squared*

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Intel Corporation
United States United States
You may know us for our processors. But we do so much more. Intel invents at the boundaries of technology to make amazing experiences possible for business and society, and for every person on Earth.

Harnessing the capability of the cloud, the ubiquity of the Internet of Things, the latest advances in memory and programmable solutions, and the promise of always-on 5G connectivity, Intel is disrupting industries and solving global challenges. Leading on policy, diversity, inclusion, education and sustainability, we create value for our stockholders, customers and society.
Group type: Organisation

43 members


You may also be interested in...

Pro

Comments and Discussions

 
-- There are no messages in this forum --
Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web06 | 2.8.181214.1 | Last Updated 23 May 2018
Article Copyright 2018 by Intel Corporation
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid