Intro

TorchRec is a PyTorch domain library for building and scaling recommendation systems. It provides composable building blocks for industry-scale RecSys workloads, with a focus on sparse features, embedding tables, sharding, distributed training, and inference.

Key capabilities of TorchRec include: - Embedding modules: High-performance embedding bags and tables optimized for sparse categorical features. - Sharding and parallelism: Built-in support for model, table, and row-wise sharding across GPUs and nodes. - Distributed training: Integrations with PyTorch Distributed and FSDP to train massive models efficiently. - Feature processing: Utilities for jagged tensors, pooling, and common RecSys data structures.

• TorchRec Overview
  • Why TorchRec?
• TorchRec High Level Architecture
  • TorchRec’s Parallelism Strategy: Model Parallelism
  • Embedding Tables
  • See also
• TorchRec Concepts
  • JaggedTensor
  • KeyedJaggedTensor
  • Planner
  • Sharding of EmbeddingTables
  • Distributed Training with TorchRec Sharded Modules
  • DistributedModelParallel
  • Optimizer
  • Inference
  • See Also