Databend's OpenTelemetry Exporter Turns Logs, Traces, and Metrics into SQL Queryable Tables
Storing OTel signals in a lakehouse removes the friction of hopping between separate log, trace, and metric systems. Engineers can join observability data with order tables in one SQL query, which speeds up incident response and opens the door to long-term retention and business-impact analysis without building custom pipelines.
When a payment fails, the error log is only the start. You need to follow the trace, check latency, and see which orders were affected. If logs, traces, and metrics live in separate systems, that means cross-checking Elasticsearch, Jaeger, and Prometheus. Databend's open-source OpenTelemetry exporter solves that by writing OTLP data straight into Databend Cloud as three tables: otel_logs, otel_traces, and otel_metrics. The exporter uses the native Streaming Load API, preserves raw OTLP objects in VARIANT columns, and stores timestamps as UInt64 nanoseconds to avoid precision loss.
A walkthrough on Databend Cloud runs a simulated e-commerce payment flow across four services. With about 8% of payments failing and 10% exceeding 400 ms, the demo generates 6,300 logs, 13,628 spans, and 693 metric points from 2,100 checkouts. Using SQL, you can filter error logs, reconstruct a call chain via trace_id, compute P95 latency per service, and join logs with traces to quantify lost revenue. The demo shows 177 failed orders worth $36,058.96. The same tables can be queried with BI tools, Grafana, or AI agents.
The exporter is still in development, not yet in the OpenTelemetry Collector Contrib, and uses at-least-once delivery, so duplicates are possible. It supports all five metric types, retains exemplars and histogram buckets, and allows pre-created schemas for least-privilege production setups. For teams already using Databend for analytics, this path puts observability data next to order tables, enabling incident analysis without exporting data to a separate system.
Keeping complete OTLP objects as VARIANT preserves flexibility, but forces every query to navigate nested paths, which shifts the cost of understanding data into the analyst's SQL.
An at-least-once delivery model with no deduplication might be acceptable for logs and traces but could produce inflated counts or skewed metrics in long-running production use.
The decision to keep the exporter outside Contrib until community feedback suggests the team is cautious about API stability; production users should expect changes to configuration or behavior.
The demo's artificial failure injection (8% payment failures) makes it easy to validate the workflow, but real incidents typically involve noisier data and more ambiguous root causes.