BigQuery

Learn how to set up BigQuery source connection

The GrowthLoop Audience Builder Application connects directly to BigQuery so you can leverage the customer data in your data warehouse to create audiences with just a few clicks.

To establish the connection to BigQuery, here are the key steps we will walk through together.

  1. Create a GCP Service Account
  2. Configure GCP Permissions
  3. Create a BQ dataset to store snapshots
  4. Connect your BigQuery to the GrowthLoop App

Create a GCP Service Account

The first step is to create a service account in the GCP project where your customer data lives. Navigate to the project where your customer data is, and click on IAM & Admin in the panel on the left.

Untitled

Click Service Accounts.

Untitled

Create a new Service Account by entering

  • Service account name - often company_name-flywheel
  • Service account ID - make sure you save this account ID. You will need it later.
  • Service account description

Click Create and Continue

Untitled

Next grant the service account three roles at the Project level:

  • BigQuery Job User
  • BigQuery ReadSession User
  • BigQuery Metadata Viewer

Continue to step three. Leave the Service account user role and Service account admins role fields blank and click Done.

Untitled

Navigate to the Keys tab and click Add Key.

Untitled

Click Create new key. Make sure that the JSON key type is selected and click Create. A JSON file will download to your computer. Save it in a location that can be easily found for later.

Untitled

You have now created a new Service Account and provided permissions to the GCP project that houses your customer data.

Configure GCP Permissions

Within your GCP project, navigate to the datasets that you want to connect to the app. Keep in mind that right now, all tables you are looking to join together into a dataset group need to be in the same dataset. However, you can connect tables from different datasets, as long as the joins exist within the same dataset, and the service account has permissions for each dataset they are looking to connect in the app.

Now we are going to provide read access to the datasets for the Service Account. Click on the three vertical bubbles next to a dataset that you want to connect to the application.

Untitled

Click Share. Add a new principal with the permission BigQuery Data Viewer and save. At this point, you should have a service account with four sets of permissions:

  • BigQuery Job User at the project level
  • BigQuery ReadSession User at the project level
  • BigQuery Metadata Viewer at the project level
  • BigQuery Data Viewer at the dataset level

Create BQ Dataset for Snapshots

Within the GCP project that houses your data, create a new dataset called flywheel_system. This dataset will store the snapshots created during audience exports. Grant your service account Data Owner permissions by clicking on the three vertical bubbles next to the dataset name. Click Share. Add a new principal with the permission BigQuery Data Owner and save. Now you have the flywheel_system dataset with the necessary permissions to write snapshots to BigQuery.

Connect BigQuery to the GrowthLoop Application

First, navigate to the link to the application the GrowthLoop team has provided you. If we haven’t provided a link, please reach out to the solutions architect for your project.

When you click on the link, you will be brought to the signup page. Enter your company credentials and click Sign Up.

Enter your Organization name and Industry. Click the checkbox to agree to the terms and then Continue.

Untitled

Select BigQuery as the data warehouse you wish to connect to. Click Upload file and navigate to where your service account key is located on your computer. Upload the file to the application and you will see Success appear. If you are having issues verifying your service account, please reach out to [email protected] or your solutions architect for support.

Untitled

Next, we are going to connect the applications to specific datasets in your data warehouse. Select the Dataset that you wish to connect to. All of your data that you want to bring in to the application must live in the dataset(s) you provided Data Viewer permissions to. Then, select the Table that you’d like to connect to within your dataset. Often, new users connect their Customer data table first. Later on you are also able to connect more tables that you wish to use in your analysis in the application.

Untitled

Next, we want to define our table configurations.

  1. Select the field that contains the unique id for your table. This will often be a customer id for customer data.
  2. Select a device id. This field is only important to define correctly if you plan on exporting audiences to a destination that requires device ids, like TikTok.
  3. Check the fields that you want to see automatically when you start to build a new audience. This configuration can be tweaked later, but common selections are a state or a date field.
Untitled

Click Next. Now you’re all set up and ready to build your first audiences!

Untitled

What’s Next