Google BigQuery monitoring integration | New Relic Documentation

New Relic's integrations with the Google Cloud Platform (GCP) include an integration to report Google BigQuery data to New Relic. This document explains how to activate the Google BigQuery integration and describes the data that can be reported.

Features

Google BigQuery is a data warehouse with no infrastructure to manage. With BigQuery you can specify the data schema, or allow the service to discover it for you. You can explore your data using SQL queries.

Activate integration

To enable the integration follow standard procedures to connect your GCP service to New Relic.

Polling frequency

New Relic integrations query GCP services according to a polling interval that varies depending on the integration. The polling frequency for the Google BigQuery integration is five minutes. For information about the resolution, see metric data details.

Activate Table metrics

The collection of BigQuery table metrics is disabled by default. This affects the metrics listed under GcpBigQueryTableSample. To collect those metrics, turn on the "Enable table metrics" setting under the configuration menu of the BigQuery integration.

Find and use data

After activating the integration and waiting a few minutes (based on the polling frequency), data will appear in the New Relic UI. To find and use your data, including links to your dashboards and alert settings, go to one.newrelic.com > Infrastructure > GCP > (select an integration).

Metric data

To view metric data for your Google BigQuery integration in New Relic, create NRQL queries for GcpBigQueryProjectSample, GcpBigQueryDatasetSample, and GcpBigQueryTableSample events and their attributes.

GcpBigQueryProjectSample

Query GcpBigQueryProjectSample events in New Relic to view data for the following metrics. Resolution is one data point per minute with an 8-minute delay.

Attribute	Description
`query.Count`	In-flight queries.
`query.ExecutionTimes`	Distribution of query execution times.
`slots.Allocated`	Number of slots currently allocated for project.
`slots.AllocatedForProject`	Number of slots currently allocated for query jobs in the project.
`slots.AllocatedForProjectAndJobType`	Number of slots currently allocated for the project and job type.
`slots.AllocatedForReservation`	Number of slots currently allocated for project in the reservation.
`slots.TotalAllocatedForReservation`	Number of slots currently allocated across projects in the reservation.
`slots.TotalAvailable`	Total number of slots available for the project.

GcpBigQueryDatasetSample

Query GcpBigQueryDatasetSample events in New Relic to view data for the following metrics. Resolution is one data point every 30 minutes with a 3-hour delay.

Attribute	Description
`storage.StoredBytes`	Amount of stored bytes.
`storage.Tables`	Number of tables.

GcpBigQueryTableSample

Query GcpBigQueryTableSample events to view data for the following metrics. Resolution is one data point per minute with a 6-hour delay.

Attribute	Description
`storage.UploadedBytes`	Uploaded bytes.
`storage.UploadedBytesBilled`	Uploaded bytes billed. Currently only streaming API is billed.
`storage.UploadedRowCount`	Uploaded rows.

Inventory data

EOL NOTICE

After March 2022, we're discontinuing support for several capabilities, including inventory data for cloud integrations. For more details, including how you can easily prepare for this transition, see our Explorers Hub post.

To view inventory data for Google BigQuery services, go to one.newrelic.com > Infrastructure > Inventory and search for or select the following attributes.

Data indicated with an asterisk * are only fetched when the GCP project is linked to New Relic through a service account, and when extended inventory collection is on.

gcp/bigquery/project

Name	Description
`projectId`	The ID of the project.

gcp/bigquery/dataset

Name	Description
`projectId`	The ID of the project.
`name`	The region where the instance is.
`zone*`	The location of the dataset.
`description*`	Description of the dataset.
`creationTime*`	Timestamp when the dataset was created.
`lastModified*`	Last time the dataset or any of its tables was modified.
`defaultTableLifeTime*`	Default lifetime of all tables in the dataset in milliseconds.
friendlyName*	User-friendly name for the dataset.
selfLink*	URL to access the resource.
`labels*`	Labels of the dataset.

gcp/bigquery/table

Name	Description
`projectId`	The ID of the project.
`dataSetId`	The ID of the dataset.
`name`	The ID of the table.
`zone*`	The location of the table.
`creationTime*`	Timestamp when the table was created.
`type*`	The table type.
`description*`	The description of the table.
`expirationTime*`	Timestamp when the table will expire.
`friendlyName*`	User-friendly name for the table.
`selfLink*`	URL to access the resource.
`labels*`	Labels of the table.
`lastModified*`	Time when the table was last modified.

Notes

As of November 20, 2018, for the metric storage/uploaded_bytes_billed, information is only available if data is uploaded to BigQuery via streaming.