Short name describing what triggered the graph break
Unsupported Tensor.backward() call
Values or code snippet captured at the break point
call_method {self} backward {gradient} {retain_graph} {create_graph} {inputs}
Explanation of why the graph break was triggered
Dynamo currently does not support tracing Tensor.backward() when trace_autograd_ops is off.
Hints on how to resolve the graph break
This graph break happens when Tensor.backward() is called inside a compiled
region while the option to capture backward operations is turned off (the
default). This usually appears when a whole training step (forward +
loss.backward()) is wrapped in torch.compile.
Example code that causes the graph break is:
import torch
@torch.compile(fullgraph=True)
def step(x):
loss = (x * 2).sum()
loss.backward()
return loss
x = torch.randn(4, requires_grad=True)
step(x)
The idiomatic fix is to compile only the forward computation and run
backward() outside the compiled region. torch.compile already captures the
backward pass for the operations inside the compiled forward, so you keep the
performance benefit:
import torch
@torch.compile(fullgraph=True)
def forward(x):
return (x * 2).sum()
x = torch.randn(4, requires_grad=True)
loss = forward(x)
loss.backward() # outside the compiled region
If you specifically need the backward() call to run inside the compiled
region, turn on backward-operation capture. This is an experimental option; read
the gradient (e.g. x.grad) out of the function rather than returning the tensor
you called backward() on:
import torch
torch._dynamo.config.trace_autograd_ops = True
@torch.compile(fullgraph=True)
def step(x):
loss = (x * 2).sum()
loss.backward()
return x.grad
x = torch.randn(4, requires_grad=True)
step(x)