Expand description
§Ibverbs Manager
Contains ibverbs-specific RDMA logic.
Manages ibverbs resources including:
- Memory registration (CPU and CUDA via dmabuf or segment scanning)
- Queue pair creation and connection establishment
- RDMA domain and protection domain management
- Device selection and PCI-to-RDMA device mapping
§Queue-pair lifecycle
Bringing up a queue pair to a peer is a two-sided handshake (each
side has its own QP and must learn the other side’s endpoint
before transitioning INIT → RTR → RTS). Doing all of that in
response to a single message would block our actor loop while
awaiting peer RPCs, and the peer’s symmetric request would block
waiting for us — a deadlock.
Instead, IbvManagerActor does only sync bookkeeping in the
handler and offloads the handshake to a per-QP child actor,
[QueuePairInitializer]. The store of QPs ([Self::qps]) is
keyed by [QpKey] and holds a [QpState]: Pending { info, initializer, waiters } while the handshake runs, Ready(qp)
once this side is RTS and has observed the peer’s RTS, or
Failed(error) as a tombstone after a fatal error.
Structs§
- IbvBackend
- Wrapper around
ActorHandle<IbvManagerActor>that moves the RDMA data-plane (post send/recv, poll CQ) off the actor loop while keeping state-mutating operations (MR registration/deregistration, QP management) serialized through actor messages. - IbvManager
Actor - Manages all ibverbs-specific RDMA resources and operations.
Enums§
- IbvManager
Local Message - Local-only messages for
IbvManagerActor. - IbvManager
Message - Cross-proc messages handled by
IbvManagerActor.
Traits§
- IbvManager
Local Message Client - The custom client trait for this message type.
- IbvManager
Local Message Handler - The custom handler trait for this message type.
- IbvManager
Message Client - The custom client trait for this message type.
- IbvManager
Message Handler - The custom handler trait for this message type.