Teardown#
Always kick off a barrier flush before stopping the mesh.
On child exit, tee observes EOF, forwards any partial line, and posts one last Flush(None).
StreamFwder::abort()stops the tee task and returns a peek buffer for diagnostics.FileAppendertasks self-flush on drop;LogClientActorprints any buffered aggregates on drop.
End-to-end shutdown (happy path)#
Python calls
LoggingMeshClient.flush(...)→ barrier across all forwarders (see Client & Ordering pages).Python proceeds to stop the mesh/procs.
Each child exits → bootstrap side pipes hit EOF → tee: • flushes local std writer, • forwards final partial line (if any), • calls
log_sender.flush()(Flush(None)).Bootstrap calls
StreamFwder::abort()to stop the tee tasks and join them.FileAppenderdrops → its writer tasksflushand exit.Single
LogClientActorin the Python process prints any remaining aggregates and returns from the barrier.
There is one
LogClientActorand it runs in the Python/driver process.
Component responsibilities on teardown#
Stream forwarder (bootstrap side)#
EOF handling in tee:
forwards final partial line (if present),
posts
Flush(None)once,returns
Ok(()).
// logging.rs — tee(..) tail (abridged)
std_writer.flush().await?;
if !line_buffer.is_empty() {
// forward final partial line to sender + file monitor
}
if let Some(sender) = log_sender.as_mut() {
let _ = sender.flush(); // posts LogMessage::Flush { sync_version: None }
}
Ok(())
Controller stop/join:
// logging.rs — StreamFwder::abort (abridged)
pub async fn abort(self) -> (Vec<String>, Result<(), anyhow::Error>) {
self.stop.notify_waiters();
let lines = self.peek().await; // diagnostic ring buffer
let teer_result = self.teer.await; // join spawned tee task
let result = match teer_result { Ok(inner) => inner.map_err(anyhow::Error::from), Err(e) => Err(e.into()) };
(lines, result)
}
File aggregation (bootstrap side)#
Dropping
FileAppendersignals its two writer tasks to finish and flush.
// logging.rs — FileAppender::drop
fn drop(&mut self) {
self.stop.notify_waiters();
tracing::debug!("FileMonitor: dropping, stop signal sent, tasks will flush and exit");
}
Forwarder (child side)#
No special teardown handler; when the proc exits, its served channel closes and the bootstrap side EOF path runs.
Periodic
Flush(None)heartbeats are generated during runtime (see Forwarder internals); they don’t affect teardown.
Failure paths & what you see#
Child crashed / channel closed early
LocalLogSender::send/flushchecksTxStatus. If inactive, it skips sending and logs a debug line. Teardown proceeds; you may miss the very last batch for that proc.Barrier never completes Usually
expected_procs< actual forwarders mismatch. Re-check the count you passed intoStartSyncFlush(Python’sflush()computes this for us). See Client and Ordering.Partial last line Safe:
teeforwards the remaining bytes as a final line on EOF.Interleaved stdout/stderr Expected. Ordering is per proc, per stream only (see Ordering).
Local files look incomplete If running in
Env::LocalandHYPERACTOR_FORCE_FILE_LOGis false, file aggregation may be disabled. See Config & env.