File aggregation#

What it is#

An optional, local file sink that mirrors each child’s stdout/stderr into per-proc files on the bootstrap host. It’s implemented by FileAppender plus a tiny writer task (file_monitor_task) that receives lines over a hyperactor channel and appends them to disk.

When files are created (env gating)#

Files are only created when:

Env is not Local, or
Env is Local and HYPERACTOR_FORCE_FILE_LOG=true

If the gate returns None, file aggregation is simply disabled; streaming to the client and teeing to the bootstrap console still work.

Naming scheme & paths#

Files live under the path returned by log_file_path(env, None) and are named:

{basename}_{host-tag}_{shortuuid}.{stdout|stderr}

Where:

basename — from log_file_path (per environment),
host-tag — the bootstrap host, from hostname::get(),
shortuuid — ShortUuid::generate() per FileAppender,
suffix — “stdout” or “stderr”.

// logging.rs — create_unique_file_writer (abridged)
let (path, filename) = log_file_path(env, None)?;
let mut full_path = PathBuf::from(&path);
let uuid = ShortUuid::generate();
let suffix = match output_target { Stderr => "stderr", Stdout => "stdout" };
full_path.push(format!("{}_{}_{}.{}", filename, file_name_tag, uuid, suffix));
// open(..., create/append) → tokio::fs::File

How bytes reach the files#

StreamFwder’s background tee(...) task frames lines and posts them to the file appender’s channel as FileMonitorMessage { lines: Vec<String> }. The file_monitor_task writes each line + newline and flushes.

// logging.rs — file_monitor_task (abridged)
loop {
    tokio::select! {
        msg = rx.recv() => match msg {
            Ok(FileMonitorMessage { lines }) => {
                for line in &lines {
                    writer.write_all(line.as_bytes()).await?;
                    writer.write_all(b"\n").await?;
                }
                writer.flush().await?;
            }
            Err(e) => { tracing::debug!("channel error: {e}"); break; }
        },
        _ = stop.notified() => { break; }
    }
}
// final flush on exit

Lines may already be truncated to 4 KiB upstream in tee(...) (with “… [TRUNCATED]”), and may carry an optional [rank] prefix if HYPERACTOR_PREFIX_WITH_RANK=true

Using FileAppender (bootstrap side)#

Creating an appender spins up two writer tasks (stdout/stderr) and exposes one channel address per stream:.

// logging.rs — FileAppender::new (abridged)
let app = FileAppender::new();                // Option<FileAppender>
let stdout_addr = app.as_ref().map(|a| a.addr_for(OutputTarget::Stdout));
let stderr_addr = app.as_ref().map(|a| a.addr_for(OutputTarget::Stderr));

// StreamFwder::start_with_writer(...) receives `file_monitor_addr` per stream.
// The tee task dials once and posts FileMonitorMessage batches as it frames lines.

Dropping the appender triggers a graceful stop & flush:

impl Drop for FileAppender {
    fn drop(&mut self) {
        self.stop.notify_waiters();           // writer tasks flush and exit
    }
}

Rotation semantics & guarantees#

•   Rotation: not implemented (today each `FileAppender` creates fresh, uniquely-named files).
•   Sync & durability: every batch is `flush()`ed by the writer task; a final `flush` happens on task exit and on drop.
•   Line ordering: per-stream (stdout vs stderr are independent).
•   Failure tolerance: if the writer channel closes, the task logs and exits; other paths (streaming, console tee) continue.