API Reference#
- Process Groups
ErrorSwallowingProcessGroupWrapperFakeProcessGroupWrapperManagedProcessGroupProcessGroupProcessGroupBabyProcessGroupBabyGlooProcessGroupBabyNCCLProcessGroupBabyXCCLProcessGroupDummyProcessGroupGlooProcessGroupNCCLProcessGroupWrapperProcessGroupXCCLcreate_store_client()trigger_nccl_fr_trace_through_pipe()- Manager
ExceptionWithTracebackManagerWorldSizeModeextract_trailing_digits()get_timeout()- Optimizers
OptimizerWrapper- Distributed Data Parallel
DistributedDataParallelPureDistributedDataParallel- LocalSGD
DiLoCoLocalSGDextract_local_tensor()- Data
DistributedSampler- Checkpointing
CheckpointTransportHTTPTransport- Parameter Servers
ParameterServer- Coordination (Low Level API)
LighthouseClientLighthouseServerManagerClientManagerServerQuorumQuorumMember
