torchcodec.decoders

Video Decoding

For a video decoder tutorial, see: Decoding a video with VideoDecoder.

The VideoDecoder class is a high-level entry point for video decoding. For low-level control over the decoding process, see Low-level decoding APIs for video and audio.

VideoDecoder

A single-stream video decoder.

VideoStreamMetadata

Metadata of a single video stream.

CUDA decoding utils:

set_cuda_backend	Context Manager to set the CUDA backend for VideoDecoder.
set_nvdec_cache_capacity	Set the maximum number of NVDEC decoders that can be cached (per GPU).
get_nvdec_cache_capacity	Get the capacity of the per-device NVDEC decoder cache.

CpuFallbackStatus

Information about CPU fallback status.

Audio Decoding

For an audio decoder tutorial, see: Decoding audio streams with AudioDecoder.

The AudioDecoder class is a high-level entry point for audio decoding. For low-level control over the decoding process, see Low-level decoding APIs for video and audio.

AudioDecoder	A single-stream audio decoder.
WavDecoder	A fast decoder for WAV audio files.

AudioStreamMetadata

Metadata of a single audio stream.

Image Decoding

decode_image	Decode an image into a [N]CHW tensor, detecting the format automatically.
decode_jpeg	Decode a JPEG image into a CHW tensor, on CPU or CUDA.
decode_png	Decode a PNG image into a CHW tensor.
decode_webp	Decode a WebP image into a [N]CHW tensor.
decode_gif	Decode a GIF image into a [N]CHW tensor.
decode_avif	Decode an AVIF image into a [N]CHW tensor.
decode_heic	Decode an HEIC/HEIF image into a [N]CHW tensor - requires libheif!

ImageReadMode

Color mode for image decoding.

Low-level decoding APIs for video and audio

Important

The low-level APIs are in beta. Their signatures and semantics may still change slightly, in response to user feedback. Please share your feedback!

These APIs expose the three stages of decoding as separate objects: demuxing, decoding and conversion. Reach for them when you need to control a stage, skip one, or run them on different threads. If you just want RGB frames out of a file, VideoDecoder and AudioDecoder are easier to use.

For tutorials, see:

• Build your own decoding pipeline

• Multi-threaded decoding pipelines

• Raw frames and raw audio samples

• Low-level APIs and CUDA streams synchronization

Demuxing

Demuxer	Reads one or more video and audio streams from a container, and produces their compressed Packets.
VideoStream	A video stream followed by a Demuxer.
AudioStream	An audio stream followed by a Demuxer.

get_container_metadata

Describe a container and every stream in it, without decoding anything.

Decoding

VideoPacketDecoder	Decodes the compressed Packets of one video stream into RawFrames.
AudioPacketDecoder	Decodes the compressed Packets of one audio stream into RawAudioSamples.

Conversion

ColorConverter	Turn a RawFrame (typically YUV) into an RGB Frame.
AudioConverter	Turn RawAudioSamples into normalised float32 AudioSamples, optionally resampling and remixing channels.

Data types

Packet	One compressed packet of one stream, as a Demuxer produced it.
RawFrame	One decoded video frame, exactly as the decoder produced it.

FrameIndex	What the packets of a video stream say about its frames, as returned by a scan (VideoStream.scan()).
RawAudioSamples	One decoded audio frame's samples, exactly as the decoder produced them.

Metadata

DemuxerMetadata	Container-level metadata, as reported by the header.
ContainerMetadata	Metadata of a container and of every stream in it.
VideoStreamHeaderMetadata	Metadata of a single video stream, as reported by the container header.
AudioStreamHeaderMetadata	Metadata of a single audio stream, as reported by the container header.
StreamMetadata	Metadata of a single stream, as reported by the container header.