View Custom Metrics  | Cloud Monitoring  | Google Cloud (2023)

A custom metric is any metric thatit is notdefined by Google Cloud. They include metrics you can define and include metrics defined by a third-party application. Custom metrics allow you to capture application-specific data or client-side system databuilt-in measurescollected by Cloud Monitoring can give you information about background latency or disk usage, but they can't tell you, for example, how many background routines your application has run. You can also create metrics based on the content of log entries. For information about these types of metrics, seeOverview of log-based metrics.

Custom metrics are sometimes called custom metrics or application-specific metrics. These metrics allow you or a third-party application to define and collect information that Cloud Monitoring's built-in metrics cannot. You capture such metrics using the API provided by the library to instrument your code, and then send the metrics to a backend application such as Cloud Monitoring.

You can create custom metrics directly using the Cloud Monitoring API. However, we recommend that you useOpenTelemetry. See the following documents for information on creating custom metrics:

• Purchase OTLP metrics and tracesdescribes how to use the Ops Agent and the agent's OpenTelemetry Protocol (OTLP) receiver to collect metrics and traces from applications instrumented with OpenTelemetry and running on Compute Engine.

• Google Cloud Managed Service do Prometheusadescribes how to collect Prometheusmetrics from applications running on Google Kubernetes Engine and Kubernetes.

• Saml Prometheus-metricdescribes how to use the Ops agent to collect Prometheus metrics from applications running on Compute Engine.

• Create custom metrics using the APIdescribes how to create metrics using the Cloud Monitoring API and how to add metric data to those metrics. This document illustrates how to use the Monitoring API with usage examplesAPI Explorer, C#, Go, Java, Node.js, PHP, Python and Ruby programming languages.

• Create custom metrics on Cloud Runshows how to use the OpenTelemetry Collector as a back-up agent in Cloud Run deployments.

  (Video) Logging & Monitoring - Creating and Monitoring Custom Metrics

For cloud monitoring, you can use custom metrics like built-in metrics. You can map them, set alerts about them, read them, and otherwise monitor them. For information on reading metrics, see the following documents:

• Specify metrics and resource typesexplains how to display and examine your custom and built-in metric types. For example, you can use the information in this document to list all custom metric descriptors in your project.
• Get time series dataexplainshow for retrieving time series data from metrics using the tracking API. For example, this document describes how you can use the API to get CPU utilization for VM instances in your Google Cloud project.

The Google Cloud Console has a dedicated page that helps you see your usage with custom metrics. For information about the content of this page, seeSe metrics-diagnostik.

Metric descriptors for custom metrics

Each metric type must have a metric descriptor that defines how the metric data is organized. The metric descriptor also defines the tags for the metric and the metric name. for examplemetric sheetshow metric descriptors for all built-in metric types.

Cloud Monitoring can create a metric descriptor for you using the metrics you write, or you can explicitly create a metric descriptor and then write the metrics. In either case, you need to decide how you want to organize your metrics.

Design example

Suppose you have a program running on one machine and that program calls utility programsUiB. You want to count how often programsUiBcalled. You also want to know when the program isUit is called more than 10 times a minute and the program comes upBit is called more than 5 times per minute. Finally, suppose you have a Google Cloud project and plan to write data to itgloballymonitored resource.

This example describes several different designs you can use for your custom metrics:

• Use two metrics:Metric-tip-Aprogramming call countUiMetric Type-Bprogramming call countB. In this case,Metric-tip-Acontains 1 time series, iMetric Type-Bcontains 1 time series.

  You can create one alert rule with two conditions, or you can create two alert rules each with one condition with this data mode. An alert policy can support multiple conditions, but has a single configuration for notification channels.

  (Video) Custom metrics with OpenTelemetry

  This model may be appropriate when you are not interested in data similarities between tracked activities. In this example, activities are the number of program callsUiB.

• You use a single metric and tag to store the app ID. For example, a tag can store a valueUorB.Monitoring creates a time series for each unique combination of tags. So there is a time series whose value is labelUand the second time series whose value is the labelB.

  As with the previous model, you can create one alert rule or two alert rules. However, the requirements for a whistleblowing policy are more complex. A condition that generates an event when the frequency of calls to the programUexceeds the threshold must use a filter that includes only data points whose value is labelU.

  The advantage of this model is that it is easy to calculate the ratios. For example, you can determine how much of the total is spent on callsU.

• Use a single metric to count the number of calls, but do not use a tag to record which program was called. In this model, there is one time series that combines the data for the two programs. However, you cannot create an alert rule that meets your goals because the data for the two programs cannot be separated.

The first two designs allow you to meet your data analysis requirements; however, the final design does not.

For more information seeCreate custom metrics.

Names of custom metrics

When you create a custom metric, you define a string ID that represents the metric type. This string must be unique among user-defined metrics in your Google Cloud project and must use a prefix that designates the metric as a user-defined metric. Prefixes are allowed for trackingcustom.googleapis.com/,external.googleapis.com/user, iexternal.googleapis.com/prometheus. The prefix is ​​followed by a name that describes what you are collecting. For details on the recommended way to name metrics, seeMetric Naming Rules.These are examples of two types of identifiers for metric types:

custom.googleapis.com/cpu_utilization custom.googleapis.com/instance/cpu/utilization

In the previous example, the prefixcustom.googleapis.comindicates that both metrics are custom metrics. Both examples refer to metrics that measure CPU utilization; however, they use different organizational models. When you expect to have a large number of custom metrics, we recommend using a hierarchical naming structure like the one used in the second example.

All metric types have globally unique identifiers calledresource names. The resource name structure for a metric type is:

project/PROJECT_ID/metricDescriptors/METRIC_TYPE

whereMETRIC_TYPEis a string identifier of metric type. If previous metric examples were created in the projects ID-project, then their resource names for these metrics would be:

projects/my-project-id/metricDescriptors/custom.googleapis.com/cpu_utilization projects/my-project-id/metricDescriptors/custom.googleapis.com/instance/cpu/utilization

Name or species?It is in the metric descriptorTo dofield stores the resource name of topic type itipthe field savesMETRIC_TYPEcable.

Monitored resource types for custom metrics

When entering your data into a time series, you need to specify where the data comes from. To specify a data source, select a monitored source type that represents where your data comes from, and then use it to describe a specific origin. The monitored resource is not part of a metric type. Instead it isweather seriesto which you write data includes a metric type reference and a monitored resource reference. The metric type describes the data, while the monitored resource describes where the data comes from.

Consider the monitored resource before creating your metric descriptor. The type of monitored resource you are using affects which tags you include in the metric descriptor. for exampleResurs Compute Engine VMcontains labels for the project ID, instance ID, and instance zone. Therefore, if you plan to write your metrics against a Compute Engine VM resource, then the resource tags include the instance ID, so you don't need the instance ID tag in the metrics descriptor.

Each data point of your metric must be associated with a monitored resource object. Points from different monitored resource objects are stored in different onesweather series.

You must use one of the following monitored resource types with custom metrics:

• aws_ec2_instance: Amazon EC2 instanca.
• data_flow_job: Dataflow job.
• home_station: App Engine instanca.
• gce_instance: Compute Engine instanca.
• generic_node: user-specified computing node.
• generic_task: Custom task.
• gke_container: GKE container instance.
• globally: Use this resource when no other type of resource is suitable. For most use cases,generic_nodeorgeneric_taskis a better choice thanglobally.
• k8s_clusters: Kubernetes cluster.
• k8s_container: Kubernetes container.
• k8s_node: Kubernetes node.
• k8s_pod: Kubernetes pod.

Common practice is to use monitored resource objects that represent the physical resources on which your application code runs. This approach has several advantages:

• You get better performance compared to using one type of resource.
• You avoid out-of-order data caused by multiple processes writing the same time series.
• You can group your custom metrics with other metrics from the same sources.

globallyand generic resources

Thegeneric_taskigeneric_noderesource types are useful in situations where none of the more specific resource types are suitablegeneric_tasktype is useful for defining task-like resources such as applications. Yesgeneric_nodetype is useful for defining node-like resources such as virtual machines. Bothgeneric_*types have several common tags that you can use to define unique resource objects, making them easy to use in aggregation and reduction metric filters.

In contrastgloballytype of resource has onlyproject_idiPlacelabels. When you have many metrics sources in a project, use the same onesgloballyresource object can cause collisions and delete your metrics.

API methods that support custom metrics

The following table shows which methods in the Tracking API support custom metrics and which methods support built-in metrics:

Limitations and delays

For restrictions on custom metrics and data retention, seeQuotas and limits.

To retain your metrics after the retention period, you must manually copy the data to another location, such as Cloud Storage or BigQuery.

For information about the delays associated with writing data to custom metrics, seeMetric data latency.

What's next

• UseGoogle Cloud Managed Service do Prometheusato collect Prometheusmetrics from applications running on Google Kubernetes Engine and Kubernetes.
• Saml Prometheus-metricfrom applications running on Compute Engine.
• Purchase OTLP metrics and tracesfrom applications instrumented with OpenTelemetry and running on Compute Engine.
• Create custom metrics using the API
• Uvod u Cloud Monitoring API
• Metrics, Time Series and Resources