Module handle 

Handle: an observe-only handle to a background operation.

A Handle wraps a watch channel that a producer fulfills exactly once with the operation’s PyResult<Py<PyAny>>. You observe its eventual result: synchronously via get(), on an asyncio loop via as_asyncio()/await, or without blocking via poll(). The channel is multi-observer, so a resolved value stays observable by any number of later observers.

The watch-channel mechanics live in [HandleCore].

Handle invariants (HDL-*)

Code that relies on or enforces an invariant is tagged // HDL-N, and each #[cfg(test)] test that attests one names it; read the matching entry below before changing a tagged site.

Assumed of the shared core and producer (not enforced here):

• HDL-1 (single terminal completion). The watch value goes None -> Some(_) exactly once and is never reset (Some -> Some, Some -> None). Relied on by [HandleCore::wait_future] (wait loop + expect), [HandleCore::poll] (re-borrow across the GIL), and PyHandle::wait_completion (discriminant check then GIL re-borrow). Held by convention: every producer sends once, always Some(_) (including the direct PyHandle::spawn producer). A send-once sender (a newtype consuming the watch::Sender) could make it structural, hardening poll’s re-borrow.
• HDL-2 (value encoding). None is pending, Some(Ok(_)) success, Some(Err(_)) a producer error.

Guaranteed by this module:

• HDL-3 (non-consuming reads). No read moves the value out: a value-extracting read clones it (clone_ref), and a discriminant-only read (wait_completion’s success fast path) borrows just the Ok/Err tag and clones nothing. Either way any number of observers see the same result. Enforced by construction: no read path moves the value.
• HDL-4 (drop does not cancel; drop never panics). Dropping a Handle never cancels its producer; a Handle observes, it does not own the producer’s lifecycle. Enforced by construction: a Handle’s core has abort_on_drop: None, and HandleCore::Drop aborts only when abort_on_drop is set. The abort is wrapped in catch_unwind, so an abort() that panics at interpreter shutdown (the Tokio runtime already gone) cannot escape Drop and abort the process.
• HDL-5 (GIL discipline). No Python object is touched without the GIL, and every acquisition goes through monarch_with_gil{,_blocking}. Enforced by the crate’s #![deny(clippy::disallowed_methods)] ban on raw Python::with_gil/attach – a hard compile error, not a debug_assert.
• HDL-6 (WouldBlockRuntime is get()’s alone among the observers). Of the Handle observers only get() raises it, and only in a Tokio runtime context; as_asyncio()/__await__ off a loop raise the native RuntimeError instead. The exception type itself is not private to this module: it marks synchronous APIs that refuse to enter or block on Tokio from an existing Tokio runtime context. The Python layer also raises it for fresh root-client bootstrap (monarch._src.actor.actor_mesh), the blocking worker-loop wrappers (monarch._src.actor.bootstrap), and a returns_future .get() that would block a running event loop (monarch._src.actor.returns_future, RF-7).
• HDL-7 (as_asyncio publish). The observer waits borrow-first (via wait_ready, never changed()-first). The result is published on the loop thread through complete_asyncio_future, by the wake reader (HDL-17) or by the fallback’s call_soon_threadsafe callback (HDL-21), or, for a Handle already complete, by as_asyncio itself (HDL-22). It sets a result only on a non-cancelled future, swallowing InvalidStateError; any StopIteration (including subclasses) is wrapped in RuntimeError first, since set_exception rejects StopIteration (PEP 479). If the loop has closed, the fallback logs rather than panics and the wake reader never runs. A wake-reader failure, including a future whose done() raises, is reported through the future’s loop exception handler, as asyncio reports an exception from a callback.
• HDL-8 (get() loop-warning is call-pattern, not outcome). get() on a running asyncio loop warns regardless of whether the value is ready: it flags the blocking-call-from-a-loop anti-pattern, not whether this call happens to block. A ready value returns after warning; under a warnings-as-errors filter the warning escalates to an error.
• HDL-9 (GIL/watch borrow discipline). A watch borrow is never held across a GIL acquisition or an .await. poll and wait_completion’s error path drop their read guard before taking the GIL and re-borrow under it (GIL-then-watch order, never the inverse against the producer’s write-lock send); the wait_ready/ wait_future loops drop the temporary Ref before each .await; and their returned futures own a cloned receiver (Send + 'static, borrowing nothing from &self), which is what lets get() drop(slf) + release the GIL before blocking and lets as_asyncio spawn the observer. as_asyncio’s fallback observer clones the value under the one GIL acquisition it makes to schedule; its wake observer never reads the value (HDL-17). Enforced partly by the compiler (Send bounds, the borrow checker on drop(slf)) and partly by construction; exercised by the multi-observer/blocking tests.
• HDL-10 (dropped producer surfaces, never hangs). If every producer drops its sender without sending, the wait loop’s changed().await? yields a RecvError converted to RuntimeError("Handle producer ended without publishing a result") on the non-blocking (poll), sync (get/wait_future), async (as_asyncio), and wait_completion paths, so every observer receives the same producer-lifecycle failure rather than waiting forever.
• HDL-11 (Python construction is paired). Neither Handle nor _HandleCompleter has #[new]. Internal Python may call the private _new_handle_pair(), which atomically creates a pending Handle and its sole producer. The factory initializes the completer with Some(sender). set_exception() rejects a non-BaseException before taking it; the first set_result() or valid set_exception() takes it, producing the only state transition to None, and any later completion attempt raises. No sender clone escapes. Dropping the completer unresolved closes the channel and surfaces through HDL-10. Python therefore cannot create a pending Handle without its producer. Native producers remain the default; this private pair is reserved for operations whose semantics and scheduling are owned by Python.
• HDL-12 (get() timeout contract). get() validates the timeout up front (rejecting negative/NaN/non-finite as ValueError via try_from_secs_f64, never panicking) and before the ready fast path, so an invalid timeout raises deterministically even for a ready handle. A timeout is non-cancelling: it raises TimeoutError while leaving the producer and every observer untouched, so a later poll()/get()/await still observes completion.
• HDL-13 (direct Rust producer). A Rust caller can eagerly produce a Handle from a Send + 'static future via PyHandle::spawn, without a PyPythonTask: it spawns one producer that converts the terminal value to Py<PyAny> under monarch_with_gil and publishes it once through the shared send_result helper (same one-shot core and unobserved-error policy as pytokio’s producers, HDL-1/HDL-2), with abort_on_drop: None (HDL-4). PyHandle::wait_completion is the matching observer: it projects the terminal value to PyResult<()>, touching neither the GIL nor the Python value on the ready-success fast path (a discriminant-only read, HDL-3) and cloning the PyErr under the HDL-9 borrow order only on the error path. Both are pub Rust-only methods outside #[pymethods] (HDL-11).
• HDL-14 (Rust value observation is owned and non-consuming). PyHandle::wait_future returns a Send + 'static future over a cloned watch receiver. The waiter borrows nothing from the PyHandle, survives the wrapper being dropped, and leaves the terminal value available to every other observer.
• HDL-15 (Rust ready construction starts no work). PyHandle::from_value constructs an already-complete, non-aborting core without spawning a task. It is public to Rust callers but remains outside #[pymethods], preserving HDL-11’s direct-construction restriction.
• HDL-16 (permanent Python identity). Handle and WouldBlockRuntime have the canonical module monarch._rust_bindings.monarch_hyperactor.handle. The legacy pytokio module does not export Handle.
• HDL-17 (GIL-free completion wake). For a Handle still pending when as_asyncio is called, on a loop that accepts add_reader, the observer task holds a Notifier (a token, an mpsc sender and an Arc<Waker>) and the wait_ready future. It publishes by pushing the token and writing the loop’s pipe, never reading the value and taking no GIL; the loop thread reads the value from the Handle and sets it (see crate::handle_wake). Enforced by construction: Notifier has no Py<_>, and the task body makes no monarch_with_gil call.
• HDL-18 (notification ordering under supported pipe outcomes). A notifier pushes its token before writing the pipe, and the reader drains the pipe before the queue, never returning between the two; both retry EINTR. A write that succeeds or returns EAGAIN leaves a byte pending, so a token pushed after a queue drain is delivered by a later wake. EPIPE or a disconnected queue means the channel is gone and the token is dropped. Any other errno is logged as a broken wake channel, not claimed as delivered.
• HDL-19 (loop not retained). The wake registry holds each loop as a weak key and its channel through a weak reference; the loop’s selector, through the channel’s reader callback, is the channel’s only strong owner. A channel holds no reference to its loop. It holds each pending await’s asyncio future strongly until delivery, cancellation or loop close, because asyncio holds tasks weakly and that reference is the waiting task’s only owner outside the task’s own cycle; __traverse__ exposes each entry’s future and Handle object to the garbage collector. Closing a loop therefore releases the channel-owned graph (the channel, its read end, its entries, their futures and Handles). The write end stays open until every Notifier is dropped, so a loop closed while an await waits on a Handle that never completes keeps that descriptor (HDL-20). A loop dropped without being closed while an await on a pending Handle is registered is still freed by the collector. Not freed: a loop dropped without being closed while an undelivered entry’s Handle value references it, and any independent cycle through a Handle, such as a value that references its own Handle. Both run through PyHandle, which the garbage collector does not traverse.
• HDL-20 (late wake is harmless). A notify after its channel is gone is dropped without error or panic: the send fails, or the write gets EPIPE. This requires SIGPIPE to be ignored, as Rust’s runtime and CPython set it by default; in a process that restores SIG_DFL, a completion that races loop teardown can kill the process. Every Notifier owns the write end through its Arc<Waker> until its Handle completes, so no write reaches a reused descriptor.
• HDL-21 (fallback). For a Handle still pending, a loop whose add_reader raises NotImplementedError or that cannot be weakly referenced uses the call_soon_threadsafe path; HDL-7 applies there unchanged.
• HDL-22 (ready shortcut). as_asyncio() on a Handle that already holds its outcome, a value, a stored exception or producer loss (HDL-10), returns a loop future already completed with it through complete_asyncio_future, with no observer task and no wake-channel registration. Awaiting it returns without giving the loop a turn, as for any finished asyncio future.

Structs

PyHandle

The observe-only handle to a background Tokio task or Python-owned operation.

WouldBlockRuntime

raised when a synchronous API refuses to enter or block on Tokio from an existing Tokio runtime context – Handle.get(), fresh root-client bootstrap, or a blocking worker-loop wrapper – or when a returns_future .get() would block a running event loop

Functions

after_ready

Run operation only after ready resolves Ok, returning the readiness error unchanged if it does not.

register_python_bindings

Register the permanent Handle Python bindings.

to_py_error

Convert a Rust error into the generic Python ValueError used by Monarch’s native bindings.