Google Cloud Storage

 Introduction

Google Cloud Storage is an enterprise public cloud storage platform that can house large unstructured data sets.

MoEngage <> GCS

The MoEngage and GCS integration exports Campaign Interaction Events to your Google Cloud Storage buckets. You can then import these files into BigQuery for further processing and analysis.

Integration 

library_add_check

Prerequisites

Ensure you have a Google Cloud Platform account with permission to create a new bucket and a service account.

Step 1 - Create a dedicated service account for MoEngage

On your Google Cloud Platform console, go to the IAM & Admin console.

  1. From the sidebar, open "Service Accounts".

serviceaccountshome.png

  1. Click on the "+ Create Service Account" button.
  2. Give this service account a name, and an optional description.
  3. Click on "Create and continue".

newserviceaccount.png

  1. Then add the role of "Basic >> Browser" to this service account.

browserrole.png

 

  1. Click "Continue". Optionally, add users' access to this service account.
  2. Click "Done".
  3. Open the service account from the list of service accounts.
  4. Go to the "Key" tab, and then click on "Add Key". Select "Create new key".

newkey.png

  1. Select the Key Type as JSON. Then click "Create".

jsonkey.png

  1. Your new key will be automatically downloaded in JSON format. Make sure to store this safely. You will need to upload this later on the MoEngage dashboard.

Step 2: Grant this service account permissions to your bucket

  1. On your service account details page, copy the email of your Service Account.

serviceaccountemail.png

  1. Go to the Cloud Storage Console, and open the bucket in which you want MoEngage to export the events.
  2. Under the Permissions tab, click on "Grant Access".

GrantBucketAccess.png

 

  1. Paste and then select the email of the service account in the "New Principals" field.
  2. Assign the roles of "Cloud Storage >> Storage Admin".

bucketpermission.png

  1. Click "Save".

Step 3: Enable APIs & Services in your GCP Account

  1. In your GCP Console, go to "APIs & Services", then click on "+ Enable APIs And Services".

EnableAPIs.png

  1. Search for "Cloud Storage" and open the API details page.
  2. Click on Enable to enable it.

enablecloudstorageapi.png

Step 4: Setup the GCS Integration on the App Marketplace

  1. On your MoEngage Dashboard, go to the App Marketplace.
  2. Search for "Google Cloud Storage".
  3. Go to the "Integrate" tab, and click on "+ Add Integration".

addintegration.png

  1. Give your connection an identifiable name.
  2. Enter your bucket name.
  3. Optionally, select your bucket region. You can find this on your Bucket Details page.
  4. Upload the Service Account Credential JSON file that you downloaded earlier. You can verify your Client ID and Client Email upon upload.
  5. Select which events you want to export. By default, each event will be exported with standard attributes.

integration1.png

  1. Optionally, you can choose to test the connection. MoEngage will attempt to verify if the details you entered are correct. If there is an issue, an error will be shown that you can rectify.

Note on required permissions

To perform end-to-end testing, MoEngage requires a few additional permissions to be configured. If your IT Policy does not allow for these permissions, you can skip them. However, it might not be possible for you to test the connection from our app marketplace. Please note that skipping these optional permissions will not impact the actual data export in any way.

<tr"> resourcemanagement.projects.get No

Allows the user to view metadata for a project in Google Cloud Resource Manager.

MoEngage uses this permission to access your service account during testing.

</tr">

Permission Required Description
resourcemanagement.projects.getIampolicy No

Allows the user to view the IAM policy for a project in Google Cloud Resource Manager.

MoEngage uses this permission to check whether you have assigned the right policy to your bucket.

storage.bucket.get No

Allows the user to view metadata for a Google Cloud Storage bucket

MoEngage uses this permission to check whether the bucket that you have provided during configuration actually exists or not.

storage.object.create Yes

Allows the user to create objects (files) in a Google Cloud Storage bucket.

MoEngage requires this permission to create files in your bucket.

storage.object.get Yes

Allows the user to read the contents of objects (files) in a Google Cloud Storage bucket.

MoEngage requires this permission to verify if the files are correctly created in your bucket.

storage.object.update Yes

Allows the user to modify the contents of objects (files) in a Google Cloud Storage bucket.

MoEngage requires this permission to update files in case of any discrepancies found.

 

Feel free to create a custom role that incorporates the mandatory permissions required by MoEngage and other optional permissions to aid you in testing as cleared by your IT Policies.

Verify if you are receiving the data correctly

MoEngage will upload files in the following structure:

gs://<your-bucket-name>/event-exports/<your_app_name>/<connection_id>/export_day=YYYY-MM-DD/export_hour=HH/<file-name>.json.gz

Go to your bucket, and verify if the files are present:

verify.png

Export Frequency

We will dump the data at hourly intervals.

Sample File

The exported files will be in the .json.gz format. Click here to download a sample file. Please note that this file will differ based on the events and attributes you opt to export.

 

JSON

    {
    "app_name": "App Name",
    "export_hour": "6",
    "db_name": "MoEngage App Name",
    "event": {
        "uid": "user id",
        "event_type": "CAMPAIGN_EVENT",
        "event_code": "MOE_EMAIL_SENT",
        "event_name": "Email Sent",
        "user_attributes": {
            "moengage_user_id": "userid1",
            "user_attr1": "value1",
            "user_attr2": "value2"
        },
        "event_source": "MOENGAGE",
        "email_id": "test@xyz.com",
        "event_attributes": {
            "campaign_name": "Campaign name",
            "moe_campaign_tags": [
                "Tag1",
                "Tag2"
            ],
            "campaign_type": "GENERAL",
            "moe_campaign_channel": "Email",
            "moe_delivery_type": "One Time",
            "campaign_id": "moengage campaign id",
            "campaign_channel": "EMAIL"
        },
        "event_uuid": "3b918064-08dc-47ae-b674-c23797475a8d",
        "event_time": 1653022310,
        "device_attributes": {
            "moengage_device_id": "device id if applicable"
        }
    },
    "export_day": "2022-05-20"
}

For a list of updated events and attributes, refer to our Data Export Glossary.

Was this article helpful?
2 out of 3 found this helpful

How can we improve this article?