Snowflake

Introduction

Snowflake is a data platform that allows organizations to store, analyze, and process large volumes of structured and semi-structured data in a highly scalable and efficient manner. With its unique architecture, Snowflake allows organizations to consolidate their data, perform quick analytics, and gain valuable data-driven insights accessible to all users.

MoEngage <> Snowflake

The MoEngage <> Snowflake integration allows you to set up a direct connection between your Snowflake instance and MoEngage app to sync data on a recurring basis. You can define a schedule to run the sync periodically - syncs can be as frequently as every hour or as infrequently as once per month. When a sync runs, MoEngage will directly connect to your data warehouse instance, retrieve all new data from the specified table, and update the corresponding data on your MoEngage dashboard.

Benefits

Reduce integration time

  • No more searching for the right ETL tool, as MoEngage directly integrates with Snowflake.
  • Long and complicated ETL pipelines are now replaced with a one-time integration setup that gives MoEngage direct access to your data without actual data flow.
  • This decreases the dependency on tech teams significantly.

Faster data processing

  • The power of the Snowflake infrastructure enables us to store, process, and query massive amounts of data in near real-time.
  • Any changes in the original schema are propagated immediately without having to change any configuration on MoEngage’s end, which is a significant advantage over the traditional ETL pipelines.
  • Since there is no need for ETL tools and external cloud providers, the cost of import is significantly lower than traditional data pipelines.

Integration use-cases

  • Keep your users in MoEngage up-to-date with the user data in your Snowflake account.
  • Import event data from your Snowflake account directly into MoEngage.
  • Import users into MoEngage and retarget them via custom segments.

Integration

library_add_check

Prerequisites

Ensure you have a Snowflake account with enough permissions to create new users, schemas or tables.

Set up a new Snowflake Connection

In order to import or export from Snowflake, you will first need to connect MoEngage to your Snowflake data warehouse. To create a new Snowflake Connection:

  1. On your MoEngage Dashboard, go to the App Marketplace.
  2. Search for "Snowflake".

  1. Go to the "Integrate" tab, and click "+ Add Connection".
  2. Provide your Snowflake data warehouse details:

Field Description Example
Connection Name Give this connection an identifiable name. My Snowflake Prod
Snowflake Account Identifier

The account identifier of your Snowflake account.

  • It is always found at the beginning of your Snowflake URL (https://ACCOUNT_IDENTIFIER.snowflakecomputing.com).

  • The format may differ based on Snowflake account age. For details, visit Snowflake docs.

If you're using Snowsight, you can find click on your account name at the bottom of the left nav. Select the account you want to import from and then click on the “copy” icon to copy the Account Identifier. For up-to-date information, you can visit Snowflake docs.

Format should be:

ACCOUNT_IDENTIFIER.snowflakecomputing.com

Warehouse

The name of the warehouse MoEngage will use to execute the queries.

COMPUTE_WAREHOUSE
Database

The database that MoEngage should use to read the tables from. You can pick the schema while setting up imports/exports.

MY_DEMO_DB
Username

Either an existing database username with the read access or create a new one for MoEngage.

 
Authentication Method

MoEngage can connect to your Snowflake Instance using two methods:

  1. Password - Please provide the password of the database user (username) you entered above.
  2. Key - MoEngage can connect to your Snowflake Instance using the Key pair authentication method. To use this method:
    1. Select "Key" as your authentication method.
    2. Click on "Generate Key" and MoEngage will display your public key. You will need to add this public key to your database user. Follow the instructions mentioned on Snowflake help docs to set this up:
    3. Once you have added this key to your database user, click on "Test connection" and MoEngage will attempt to verify the credentials.

MoEngage also supports the rotating of keys. If your IT Policy requires to rotate the keys, you can do it by following these steps:

  1. Go to the App Marketplace >> Snowflake >> Integrate and edit the connection.
  2. Click on the generate new key icon:
  3. A confirmation will be asked for you to review. Please read the instructions carefully. Once you save the connection with the newly generated key, MoEngage will no longer use the previous key. You have to ensure that the new key is attached to your database user as the second RSA key (see Snowflake help doc on rotating keys) for the rotation to work properly. If the new key is not attached after saving the connection, your imports and exports might break:
  4. Your new key will be generated and shown to you. At this point, it is highly recommended to copy this new key and attach it as an additional key to your Snowflake database user by following the instructions provided on the Snowflake help docs. Once you have attached the new key, you need to test the connection successfully to complete the rotation.
  5. At any point after generating the new key, if you decide to close the edit form, or do not click on "Connect", your old (existing) key will continue to be used as before and the newly generated key will be deleted from our system.
 
Role

This role will be used in executing queries from MoEngage.

 
  1. You can click Test connection to test your details or directly click on Save connection to proceed.

Once you have set up a Snowflake Connection, you can use that to set up various imports and exports in MoEngage.

If your Snowflake instances aren't public and are on AWS, we recommend you set up AWS Private Link with MoEngage AWS account. Please reach out to your account manager for more information on this.

 

Import users and events from Snowflake into MoEngage

Read our Snowflake Imports guide to set up MoEngage <> Snowflake imports for your account.

Export campaign interaction events from MoEngage to Snowflake

info

Information

For the full list of events and attributes exported, refer to the Data Exports Glossary.

On your MoEngage dashboard, navigate to App Marketplace >> Snowflake.

      1. Click Integrate >> Add Integration 
      2. Select your Snowflake connection.
      3. Select which schema MoEngage will use to create new tables and export your events into. 
      4. Select the events and additional user properties you want to export to Snowflake.
      5. Click "Save Export". 

Data would now start reflecting on your Snowflake account in 15 minutes. 

Sample Table Structure

MoEngage will create the following two tables in your schema:

  1. test_connection_moengage - We will use this table to verify if your connection details are valid from time to time.
  2. moengage_events - We will dump (append) all your events in this table.

The final table definition will depend on the exact configurations of your exports.

Field Type NL
app_name character varying(256) NN
event_name character varying(256) NN
event_code character varying(256) NN
event_uuid character varying(256) NN
event_time bigint NN
event_type character varying(256) NN
event_source character varying(256) NN
uid character varying(256) NULL
user_attributes_moengage_user_id character varying(256) NULL
device_attributes_moengage_device_id character varying(256) NULL
push_id character varying(256) NULL
email_address character varying(256) NULL
mobile_number character varying(256) NULL
event_attributes_app_version character varying(256) NULL
event_attributes_sdk_version character varying(256) NULL
event_attributes_platform character varying(256) NULL
event_attributes_campaign_id character varying(256) NULL
event_attributes_campaign_name character varying(256) NULL
event_attributes_campaign_type character varying(256) NULL
event_attributes_campaign_channel character varying(256) NULL
event_attributes_readable_campaign_id character varying(256) NULL
event_attributes_parent_campaign_id character varying(256) NULL
event_attributes_parent_flow_id character varying(256) NULL
event_attributes_parent_flow_name character varying(256) NULL
event_attributes_variation_id character varying(256) NULL
event_attributes_locale_id character varying(256) NULL
event_attributes_locale_name character varying(256) NULL
event_attributes_url character varying(256) NULL
event_attributes_timestamp bigint NULL
event_attributes_first_session boolean NULL
event_attributes_logged_in_status character varying(256) NULL
event_attributes super NULL
user_attributes super NULL
device_attributes super NULL
created_at character varying(256) NULL

Export Frequency

We will dump the data at hourly intervals.

 

Previous

Next

Was this article helpful?
2 out of 4 found this helpful

How can we improve this article?