Shuffler

class torchdata.datapipes.map.Shuffler(datapipe: MapDataPipe[T_co], *, indices: Optional[List] = None)

Shuffle the input DataPipe via its indices (functional name: shuffle).

When it is used with DataLoader, the methods to set up random seed are different based on num_workers.

For single-process mode (num_workers == 0), the random seed is set before the DataLoader in the main process. For multi-process mode (num_worker > 0), worker_init_fn is used to set up a random seed for each worker process.

Parameters:

• datapipe – MapDataPipe being shuffled

• indices – a list of indices of the MapDataPipe. If not provided, we assume it uses 0-based indexing

Example

>>> from torchdata.datapipes.map import SequenceWrapper
>>> dp = SequenceWrapper(range(10))
>>> shuffle_dp = dp.shuffle()
>>> list(shuffle_dp)
[0, 4, 1, 6, 3, 2, 9, 5, 7, 8]