Welcome to the TorchCodec documentation!#
TorchCodec is a PyTorch-native library for decoding and encoding media: videos, audio, and images, on CPU and CUDA GPU. It aims to be fast, easy to use, and well integrated into the PyTorch ecosystem. If you want to use PyTorch to train ML models on videos, audio, or images, TorchCodec is how you turn these into tensors, and back.
We achieve these capabilities through:
Pythonic APIs that mirror Python and PyTorch conventions.
Relying on FFmpeg to do the video and audio decoding / encoding. TorchCodec uses the version of FFmpeg you already have installed. FFmpeg is a mature library with broad coverage available on most systems. It is, however, not easy to use. TorchCodec abstracts FFmpeg’s complexity to ensure it is used correctly and efficiently. FFmpeg is optional: the image decoders and encoders don’t need it.
Returning data as PyTorch tensors, ready to be fed into PyTorch transforms or used directly to train models.
Installation instructions#
How to install TorchCodec
Decoding#
A simple video decoding example
How to decode images (JPEG on CPU and CUDA, PNG, WebP, and more)
A simple audio decoding example
A simple example demonstrating CUDA GPU decoding
How to efficiently decode videos from the cloud
How to decode a video with multiple processes or threads.
How to sample regular and random clips from a video
How to apply transforms while decoding
Tips for optimizing video decoding performance
How to decode HDR videos with the output_dtype parameter
Encoding#
How encode audio and video streams (CPU and CUDA)
How to encode image tensors into JPEG (CPU and CUDA) or PNG
How to encode video frames into a video file
How to encode audio samples into an audio file
Migrating from torchvision#
How to port torchvision.io image decoding and encoding code