Enum PeerDeviceAffinityPolicy 

pub enum PeerDeviceAffinityPolicy {
    Any,
    MatchName,
    Groups(Vec<BTreeSet<String>>),
}

Which of a peer’s NICs a local NIC is allowed to pair with for a transfer.

Two NICs both being up does not mean the fabric carries traffic between them. For example, some fabrics may be organized into planes, where NICs can only talk to other NICs in the same plane. Instead of trying to add complex, dynamic network topology detection into monarch, it’s much easier to simply let the user tell us the topology of their network. This is configurable using the config attr. RDMA_PEER_DEVICE_AFFINITY.

Variants

Any

Every local NIC reaches every peer NIC.

MatchName

A local NIC reaches a peer NIC only if the two carry the same device name.

Groups(Vec<BTreeSet<String>>)

A local NIC reaches a peer NIC only where some group names both. A NIC no group names reaches nothing. There may be any number of groups, of any size, and they are disjoint, so a NIC belongs to at most one.

Implementations

impl PeerDeviceAffinityPolicy

pub fn pairs(&self, local: &[String], remote: &[String]) -> Vec<Option<usize>>

Pair each device in local with at most one device in remote.

Entry i is the index in remote that local[i] should use, or None when it has no partner: no NIC appears in two pairs, so the shorter list bounds how many pairs there are, and the policy can rule out the rest.

Same-named NICs pair first, no matter what the policy says. Whatever is left over pairs subject to the policy, taking the first NIC in remote it can still reach.

The leftovers are matched greedily in order, so an earlier device can take the only partner a later one could have used. Maximizing the number of pairs is likely not worth a matching algorithm here.

pub fn choose( &self, devices: &[String], max: Option<NonZeroUsize>, ) -> Vec<String>

The devices to serve one buffer from: at most max of devices, or all of them when max is None.

Which devices are dropped is decided by position, so the caller decides what “first” means by how it orders devices.

Selection logic depends on the policy:

• Self::Any: any two NICs can talk to each other, so buffers should spread across tied devices rather than always funneling through the first. choose picks a random offset into devices, then takes up to max starting from that offset, wrapping around if necessary.
• Self::MatchName: takes the first max NICs in the order in which they appear in devices. Callers should pass devices in a known, consistent order so that different processes agree on the choice.
• Self::Groups: takes one device from every group devices touches before taking a second from any of them, so a peer served by any of those groups still finds a partner here. A device no group names comes last: it reaches no peer device, so it is worth registering only once nothing else fills max.

Trait Implementations

impl Clone for PeerDeviceAffinityPolicy

fn clone(&self) -> PeerDeviceAffinityPolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PeerDeviceAffinityPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for PeerDeviceAffinityPolicy

impl FromStr for PeerDeviceAffinityPolicy

type Err = Error

The associated error which can be returned from parsing.

fn from_str(spec: &str) -> Result<Self>

Parses a string s to return a value of this type.

impl PartialEq for PeerDeviceAffinityPolicy

fn eq(&self, other: &PeerDeviceAffinityPolicy) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for PeerDeviceAffinityPolicy