Exporting Metrics to Monitoring Platforms

​

What metrics are exported?

​

Resource Metrics

​

Execution Metrics

​

Labels

​

How it works

• Metrics are pushed every 60 seconds
• Failed pushes are retried 3 times with exponential backoff
• If pushes fail 10 consecutive times, export is automatically paused to avoid noise (re-enable at any time from the dashboard)
• Credentials are stored encrypted and never returned in API responses

​

Supported destinations

• Grafana Cloud — Primary supported destination
• Datadog — Via OTLP endpoint
• Prometheus — Self-hosted with OTLP receiver enabled
• Custom — Any OTLP-compatible endpoint (New Relic, Honeycomb, etc.)

​

Setup Guide

​

Step 1: Get your platform credentials

echo -n "INSTANCE_ID:TOKEN" | base64

​

Step 2: Configure in the Cerebrium dashboard

1. In the Cerebrium dashboard, go to your project → Integrations → Metrics Export
2. Paste your OTLP endpoint from Step 1
3. Add the authentication headers from Step 1:

4. Click Save & Enable

​

Viewing Metrics

# CPU usage by app
cerebrium_cpu_utilization_cores{project_id="YOUR_PROJECT_ID"}

# Memory for a specific app
cerebrium_memory_usage_bytes{app_name="my-model"}

# Container scaling over time
cerebrium_containers_running_count{project_id="YOUR_PROJECT_ID"}

# Request rate (requests per second over 5 minutes)
rate(cerebrium_run_total[5m])

# p99 execution latency
histogram_quantile(0.99, rate(cerebrium_run_execution_time_ms_milliseconds_bucket{app_name="my-model"}[5m]))

# p99 end-to-end response time
histogram_quantile(0.99, rate(cerebrium_run_response_time_ms_milliseconds_bucket{app_name="my-model"}[5m]))

# Error rate as a percentage
rate(cerebrium_run_errors_total{app_name="my-model"}[5m]) / rate(cerebrium_run_total{app_name="my-model"}[5m]) * 100

# Average cold start time
rate(cerebrium_run_coldstart_time_ms_milliseconds_sum{app_name="my-model"}[5m]) / rate(cerebrium_run_coldstart_time_ms_milliseconds_count{app_name="my-model"}[5m])

# List all Cerebrium metrics
{__name__=~"cerebrium_.*"}

# CPU usage across all apps
cerebrium_cpu_utilization_cores

​

Managing Metrics Export

• Disable export: Toggle the switch off. The configuration is preserved — re-enable at any time without reconfiguring.
• Update credentials: Enter new authentication headers and click Save Changes. Use this when rotating API keys.
• Change endpoint: Update the OTLP endpoint field and click Save Changes.
• Check status: The dashboard shows whether export is connected, the time of the last successful export, and any error messages.

​

Troubleshooting

​

Metrics not appearing

1. Check the dashboard status. Go to Integrations → Metrics Export and look for the connection status. If it shows “Paused,” export was automatically disabled after repeated failures — click Re-enable after fixing the issue.
2. Run a connection test. Click Test Connection on the dashboard. Common errors:
   • 401 / 403 Unauthorized: Your auth headers are wrong. For Grafana Cloud, make sure you’re using a MetricsPublisher token (not a Prometheus Remote Write token). For Datadog, verify your API key is active.
   • 404 Not Found: The OTLP endpoint URL is incorrect. Double-check the URL matches your platform and region.
   • Connection timeout: Your endpoint may be unreachable. For self-hosted Prometheus, confirm the host is publicly accessible and port 4318 is open.
3. Check your platform’s data source. In Grafana Cloud, make sure you’re querying the correct Prometheus data source (not a Loki or Tempo source). In Datadog, check that your site region matches the endpoint you configured.

​

Metrics appear but values look wrong

• Histogram metrics have _milliseconds in the name. This is normal — Prometheus appends unit suffixes from OTLP metadata. Use the full name (e.g., cerebrium_run_execution_time_ms_milliseconds_bucket) in your queries.
• Container counts fluctuate during deploys. This is expected — you may see temporary spikes in cerebrium_containers_running_count during rolling deployments as new containers start and old ones drain.
• Gaps in metrics. Short gaps (1-2 minutes) can occur during deployments or scaling events. If you see persistent gaps, check whether export was paused.

​

Still stuck?