LLM Observability with Google Cloud and OpenLLMetry

Google Cloud, also known as Google Cloud Platform (GCP), is a cloud provider including over 150+ products and services. Among these products and services are Cloud Trace, Cloud Monitoring, and Cloud Logging which together comprise Google Cloud Observability.

Traceloop’s OpenLLMetry library enables instrumenting LLM frameworks in an OTel-aligned manner and supports writing that instrumentation data to Google Cloud, primarily as distributed traces in Cloud Trace.

Integration Instructions

Step 1. Install Python Dependencies

pip install \
   opentelemetry-exporter-gcp-trace \
   opentelemetry-exporter-gcp-monitoring \
   opentelemetry-exporter-gcp-logging \
   traceloop-sdk

Step 2. Initialize OpenLLMetry

In your application code, invoke Traceloop.init as shown:

# ...
from opentelemetry.exporter.cloud_logging import CloudLoggingExporter
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.exporter.cloud_monitoring import CloudMonitoringMetricsExporter
from traceloop.sdk import Traceloop

# ...
trace_exporter = CloudTraceSpanExporter()
metrics_exporter = CloudMonitoringMetricsExporter()
logs_exporter = CloudLoggingExporter()

Traceloop.init(
    app_name='your-app-name',
    exporter=trace_exporter,
    metrics_exporter=metrics_exporter,
    logging_exporter=logs_exporter)

Advanced Topics

Large Span Attributes

You can use the CloudTraceLoggingSpanExporter from the Google Cloud agent-starter-pack as a drop-in replacement for the CloudTraceSpanExporter. That exporter writes large attributes to Google Cloud Storage and writes a reference URL to Cloud Observability.