Crate monarch_distributed_telemetry 

Distributed Telemetry - Three-component architecture

1. DatabaseScanner (Rust): Local MemTable operations, scans with child stream merging
2. DistributedTelemetryActor (Python): Orchestrates children, wraps DatabaseScanner
3. QueryEngine (Rust): DataFusion query execution, creates ports, collects results

Data flows directly Rust-to-Rust via PortRef for efficiency.

Re-exports

pub use database_scanner::DatabaseScanner;

pub use pyspy_table::PySpyDump;

pub use pyspy_table::PySpyFrame;

pub use pyspy_table::PySpyLocalVariable;

pub use pyspy_table::PySpyStackTrace;

pub use query_engine::QueryEngine;

Modules

database_scanner

DatabaseScanner - Local MemTable operations, scans with child stream merging

pyspy_table

DataFusion table schemas for py-spy stack trace data.

query_engine

QueryEngine - DataFusion query execution, creates ports, collects results

Structs

EntityDispatcher

Dispatches entity lifecycle events to Arrow RecordBatches.

QueryResponse

Response message for streaming query results. Sent directly over Rust PortRef for efficiency. Completion is signaled by the scan endpoint returning, not via a message.

RecordBatchSink

Buffers tracing events and produces Arrow RecordBatches.

Functions

get_flush_count

Get the total number of batches flushed by counting sinks.

register_python_bindings

reset_flush_count

Reset the flush counter to zero. Useful for tests.

Type Aliases

FlushCallback

Callback function type for flushing RecordBatches. Takes ownership of the RecordBatch. The callback should handle empty batches by creating the table with the schema but not appending the empty data.