Click here to Skip to main content
14,271,759 members

Trigger Snowflake IFTTT Flows in Azure App Service

14 Aug 2019CPOL
This article shows how to automate IFTTT (if-this-then-that) workflows with standard wizards in Logic Apps.

This article is in the Product Showcase section for our sponsors at CodeProject. These articles are intended to provide you with information on products and services that we consider useful and of value to developers.

Through standards-based interfaces like OData and Swagger, the CData API Server provides a native experience in Logic Apps and Microsoft Flow with Snowflake. OData enables real-time connectivity to data; Swagger enables scaffolding, or code generation, of wizards in Logic Apps and Microsoft Flow, as well as scaffolding PowerApps. This article shows how to add Snowflake to an IFTTT (if-this-then-that) workflow in a Logic App.

Set Up the API Server

Follow the steps below to begin producing secure and Swagger-enabled Snowflake APIs:

Deploy

The API Server runs on your own server. On Windows, you can deploy using the stand-alone server or IIS. On a Java servlet container, drop in the API Server WAR file. See the help documentation for more information and how-tos.

The API Server is also easy to deploy on Microsoft Azure, Amazon EC2, and Heroku.

Connect to Snowflake

After you deploy the API Server, provide authentication values and other connection properties by clicking Settings -> Connections and adding a new connection in the API Server administration console. You can then choose the entities you want to allow the API Server access to by clicking Settings -> Resources.

To connect to Snowflake:

  1. Set User and Password to your Snowflake credentials and set the AuthScheme property to PASSWORD or OKTA.
  2. Set URL to the URL of the Snowflake instance (i.e.: https://myaccount.snowflakecomputing.com).
  3. Set Warehouse to the Snowflake warehouse.
  4. (Optional) Set Account to your Snowflake account if your URL does not conform to the format above.
  5. (Optional) Set Database and Schema to restrict the tables and views exposed.

See the Getting Started guide in the CData driver documentation for more information.

You will also need to enable CORS and define the following sections on the Settings -> Server page. As an alternative, you can select the option to allow all domains without '*'.

  1. Access-Control-Allow-Origin: Set this to a value of '*'. An Access-Control-Allow-Origin header value of '*' is required to use the API Server in the Logic Apps Designer.
  2. Access-Control-Allow-Methods: Set this to a value of "GET,PUT,POST,OPTIONS".
  3. Access-Control-Allow-Headers: Set this to "x-ms-client-request-id, authorization, content-type".

Authorize API Server Users

After determining the OData services you want to produce, authorize users by clicking Settings -> Users. The API Server uses authtoken-based authentication and supports the major authentication schemes. You can authenticate as well as encrypt connections with SSL. Access can also be restricted by IP address; access is restricted to only the local machine by default.

For simplicity, we will allow the authtoken for API users to be passed in the URL. You will need to add a setting in the Application section of the settings.cfg file, located in the data directory. On Windows, this is the app_data subfolder in the application root. In the Java edition, the location of the data directory depends on your operation system:

  1. Windows: C:\ProgramData\CData
  2. Unix or Mac OS X: ~/cdata
[Application]
AllowAuthtokenInURL = true

Access Snowflake in a Logic App

You can use the API Server in a Logic App to create process flows around Snowflake data. The HTTP + Swagger action provides a wizard to define the operations you want to execute to Snowflake. The following steps below show how to retrieve Snowflake data in a Logic App.

If your table has a column containing the creation date of a record, you can follow the steps below to write a function to check the column values for any new records. Otherwise, skip to the Create a Logic App section to send out emails to entities that match a filter.

Check for New Snowflake Entities

To find new Snowflake entities since a certain time, you can write a function that retrieves a datetime value for the start of the interval:

  1. In the Azure Portal, click New -> Function App -> Create.
  2. Enter a name and select the subscription, resource group, App Service plan, and storage account.
  3. Select your Function App and select the Webhook + API scenario.
  4. Select the language. This article uses JavaScript.
  5. Add the following code to return the previous hour in a JSON object:
module.exports = function (context, data) {
    var d = new Date();
    d.setHours(d.getHours()-1);
    // Response of the function to be used later.
    context.res = {
        body: {
            start: d
        }
    };
    context.done();
};

Add Snowflake to a Trigger

Follow the steps below to create a trigger that searches Snowflake for results that match a filter. If you created the function above, you can search for objects that were created after the start of the interval returned.

  1. In the Azure Portal, click New and in the Web + Mobile section select Logic App and select a resource group and App Service plan.
  2. You can then use the wizards available in the Logic App Designer, which can be accessed from the settings blade for the Logic App. Select the Blank Logic App template.
  3. Add a Recurrence action that will poll for the Snowflake objects. This article polls every hour. Select the timezone -- the default is UTC.
  4. Add a function action: Expand the menu in the Add Action dialog and select the option to show Azure functions in the same region. Select the Function App you created earlier and select the function that returns the interval start.
  5. Enter an empty pair of curly brackets, "{}", to pass an empty payload object to the function.
  6. Add the HTTP + Swagger action and enter the swagger URL of the API Server:

http://MySite:MyPort/api.rsc/@MyAuthtoken/$swagger

  1. Select the "Return Accounts" operation.
  2. Use the descriptions for each property to specify additional parameters such as the columns to retrieve, filters, etc. For example: Id eq '1'
    The API Server returns the descriptions and other documentation in the swagger document. You can find more information on using the OData API and supported OData in the API Server help documentation.
  3. To use the datetime value returned from the getInterval function, use the "ge" operator with a datetime column in the Accounts table and select the Body parameter in the dialog. Note that quotes must be used to surround the datetime value.
    Image 1
  4. Switch to Code View and modify the $filter expression to extract the property containing the start of the interval. Use the syntax '@{body('MyFunc')['MyProp']'.
"getAllAccounts": {
    "inputs": {
        "method": "get",
        "queries": {
            "$filter": "CreatedDate ge '@{body('getInterval')['start']}'"
        },
    "uri": "https://MySite:MyPort/api.rsc/@MyAuthtoken/Account"
}

You can now access Snowflake as data sources and destinations in your workflows.

Email New Records

Follow the steps below to email a report with any new Account entities.

  1. In the Logic Apps Designer, add an SMTP - Send Email action.
  2. Configure the necessary information for the SMTP server.
  3. Configure the From, To, Subject, and Body. You can add parameters from the Snowflake columns returned.

Click Save and then click Run to send email notifications on any Snowflake records created in the last hour.

Image 2

At this point, you have an IFFTT workflow configured in Logic Apps to report new Accounts as they are discovered in your Snowflake data warehouse.

In addition to enabling access to your Snowflake data warehouse from Azure Logic Apps, the CData API Server can be used to connect to more than 150 SaaS, Big Data, and NoSQL sources. Download a free trial of the CData API Server online and start working with all of your data in Logic Apps today.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Jerod Johnson
Technical Writer CData Software
United States United States
I'm an educator-turned-technology evangelist, with a short stint as a software developer. In all of the work I've done, data has been critical, and as businesses, industries, and services grow, I can't help but notice the growth in the breadth and depth of data usage. A common interface to data frees enterprises from the burden of connecting to their data and frees them to focus on their own business. By leveraging CData drivers to access common SQL interfaces to more than 100 SaaS, Big Data, and NoSQL sources, developers can build solid, data-driven products and analysts and data scientists can quickly and easily build insights that drive business.

While giving presentations, writing articles, engaging in webinars, and producing tutorial videos I get the opportunity to see first-hand the difference that standard connectivity makes, with regards to both the underlying data sources and the tools and apps consuming the data. Talk to me about partnering with CData to connect to your own organization's data, embedding connectivity into your data-driven solutions or building custom connectors for a new data source.

Comments and Discussions

 
-- There are no messages in this forum --
Article
Posted 14 Aug 2019

Stats

641 views
2 bookmarked