monarch_rdma::backend::ibverbs::queue_pair

Trait IbvQueuePair

pub trait IbvQueuePair:
    Debug
    + Send
    + Sync
    + 'static
    + Sized {
    // Required methods
    unsafe fn new<I: IbvDomainImpl<QueuePair = Self>>(
        domain: &IbvDomain<I>,
        config: IbvConfig,
        send_cq: Arc<IbvCq>,
        recv_cq: Arc<IbvCq>,
    ) -> Result<Self, Error>;
    fn connect(&mut self, info: &IbvQpInfo) -> Result<(), Error>;
    fn get_qp_info(&mut self) -> Result<IbvQpInfo, Error>;
    fn state(&mut self) -> Result<u32, Error>;
    fn put(
        &mut self,
        remote_dst: IbvRemoteMemoryRegionView,
        local_src: IbvMemoryRegionView,
    ) -> Result<Vec<u64>, Error>;
    fn get(
        &mut self,
        local_dst: IbvMemoryRegionView,
        remote_src: IbvRemoteMemoryRegionView,
    ) -> Result<Vec<u64>, Error>;

    // Provided method
    fn max_msg_size(&self) -> usize { ... }
}

Expand description

A NIC-backend queue pair: the unit of RDMA communication between two endpoints, and the operations a [QueuePairActor] performs on it. Each [IbvDomainImpl] names its concrete queue-pair type via IbvDomainImpl::QueuePair and builds it through Self::new.

Required Methods§

Source

unsafe fn new<I: IbvDomainImpl<QueuePair = Self>>( domain: &IbvDomain<I>, config: IbvConfig, send_cq: Arc<IbvCq>, recv_cq: Arc<IbvCq>, ) -> Result<Self, Error>

Creates a queue pair against domain in the RESET state, reporting its completions on send_cq and recv_cq; Self::connect transitions it to RTS before use.

The completion queues are created by the caller rather than here, so that one queue can back many queue pairs. Implementers hold a clone of each, which is what keeps them alive for the QP’s lifetime.

§Safety

domain’s PD (domain.as_ptr()) must be null or a valid protection domain. Callers must ensure the PD outlives the QP; the easiest way to do this is for implementers to store a clone of domain.pd() (an Arc<IbvPd>) inside the QP. send_cq and recv_cq must have been created on domain’s device context.

Source

fn connect(&mut self, info: &IbvQpInfo) -> Result<(), Error>

Transitions the QP through INIT -> RTR -> RTS, connected to info.

Source

fn get_qp_info(&mut self) -> Result<IbvQpInfo, Error>

Returns the local endpoint other peers need in order to connect to this QP.

Source

fn state(&mut self) -> Result<u32, Error>

Returns the current ibv_qp_state of the QP.

Source

fn put( &mut self, remote_dst: IbvRemoteMemoryRegionView, local_src: IbvMemoryRegionView, ) -> Result<Vec<u64>, Error>

Post an RDMA WRITE of local_src into remote_dst. The request may be chunked into multiple WRs. This method returns the list of WR ids that were posted.

Source

fn get( &mut self, local_dst: IbvMemoryRegionView, remote_src: IbvRemoteMemoryRegionView, ) -> Result<Vec<u64>, Error>

Post an RDMA READ of remote_src into local_dst. The request may be chunked into multiple WRs. This method returns the list of WR ids that were posted.

Provided Methods§

Source

fn max_msg_size(&self) -> usize

Largest transfer this QP issues as a single work request; Self::put and Self::get split anything larger into that many WRs. The owning [QueuePairActor] budgets send-queue credits against this value, so an implementation that chunks more finely must report it here or it will over-commit the send queue.