Introduction

Google Cloud Storage is an enterprise public cloud storage platform that can house large unstructured data sets.

MoEngage <> GCS

The MoEngage and GCS integration exports Campaign Interaction Events to your Google Cloud Storage buckets. You can then import these files into BigQuery for further processing and analysis.

Integration 

Step 1 - Create a dedicated service account for MoEngage

On your Google Cloud Platform console, go to the IAM & Admin console.

1. From the sidebar, open "Service Accounts".

2. Click on the "+ Create Service Account" button.
3. Give this service account a name, and an optional description.
4. Click on "Create and continue".

5. Then add the role of "Basic >> Browser" to this service account.

6. Click "Continue". Optionally, add users' access to this service account.
7. Click "Done".
8. Open the service account from the list of service accounts.
9. Go to the "Key" tab, and then click on "Add Key". Select "Create new key".

10. Select the Key Type as JSON. Then click "Create".

11. Your new key will be automatically downloaded in JSON format. Make sure to store this safely. You will need to upload this later on the MoEngage dashboard.

Step 2: Grant this service account permissions to your bucket

1. On your service account details page, copy the email of your Service Account.

2. Go to the Cloud Storage Console, and open the bucket in which you want MoEngage to export the events.
3. Under the Permissions tab, click on "Grant Access".

4. Paste and then select the email of the service account in the "New Principals" field.
5. Assign the roles of "Cloud Storage >> Storage Admin".

6. Click "Save".

Step 3: Enable APIs & Services in your GCP Account

1. In your GCP Console, go to "APIs & Services", then click on "+ Enable APIs And Services".

2. Search for "Cloud Storage" and open the API details page.
3. Click on Enable to enable it.

Step 4: Setup the GCS Integration on the App Marketplace

1. On your MoEngage Dashboard, go to the App Marketplace.
2. Search for "Google Cloud Storage".
3. Go to the "Integrate" tab, and click on "+ Add Integration".

4. Give your connection an identifiable name.
5. Enter your bucket name.
6. Optionally, select your bucket region. You can find this on your Bucket Details page.
7. Upload the Service Account Credential JSON file that you downloaded earlier. You can verify your Client ID and Client Email upon upload.
8. Select which events you want to export. By default, each event will be exported with standard attributes.

9. Optionally, you can choose to test the connection. MoEngage will attempt to verify if the details you entered are correct. If there is an issue, an error will be shown that you can rectify.

Note on required permissions

To perform end-to-end testing, MoEngage requires a few additional permissions to be configured. If your IT Policy does not allow for these permissions, you can skip them. However, it might not be possible for you to test the connection from our app marketplace. Please note that skipping these optional permissions will not impact the actual data export in any way.

Feel free to create a custom role that incorporates the mandatory permissions required by MoEngage and other optional permissions to aid you in testing as cleared by your IT Policies.

Verify if you are receiving the data correctly

MoEngage will upload files in the following structure:

gs://<your-bucket-name>/event-exports/<your_app_name>/<connection_id>/export_day=YYYY-MM-DD/export_hour=HH/<file-name>.json.gz

Go to your bucket, and verify if the files are present:

Export Frequency

We will dump the data at hourly intervals.

Sample File

The exported files will be in the .json.gz format. Click here to download a sample file. Please note that this file will differ based on the events and attributes you opt to export.

    {
    "app_name": "App Name",
    "export_hour": "6",
    "db_name": "MoEngage App Name",
    "event": {
        "uid": "user id",
        "event_type": "CAMPAIGN_EVENT",
        "event_code": "MOE_EMAIL_SENT",
        "event_name": "Email Sent",
        "user_attributes": {
            "moengage_user_id": "userid1",
            "user_attr1": "value1",
            "user_attr2": "value2"
        },
        "event_source": "MOENGAGE",
        "email_id": "test@xyz.com",
        "event_attributes": {
            "campaign_name": "Campaign name",
            "moe_campaign_tags": [
                "Tag1",
                "Tag2"
            ],
            "campaign_type": "GENERAL",
            "moe_campaign_channel": "Email",
            "moe_delivery_type": "One Time",
            "campaign_id": "moengage campaign id",
            "campaign_channel": "EMAIL"
        },
        "event_uuid": "3b918064-08dc-47ae-b674-c23797475a8d",
        "event_time": 1653022310,
        "device_attributes": {
            "moengage_device_id": "device id if applicable"
        }
    },
    "export_day": "2022-05-20"
}

For a list of updated events and attributes, refer to our Data Export Glossary.