theano - pytorch: how to directly find gradient w.r.t. loss -

in theano, easy gradient of variable w.r.t. given loss:

loss = f(x, w) dl_dw = tt.grad(loss, wrt=w) 

i pytorch goes different paradigm, you'd like:

loss = f(x, w) loss.backwards() dl_dw = w.grad 

the thing might not want full backwards propagation through graph - along path needed w.

i know can define variables requires_grad=false if don't want backpropagate through them. have decide @ time of variable-creation (and requires_grad=false property attached variable, rather call gets gradient, seems odd).

my question is there way backpropagate on demand (i.e. backpropagate along path needed compute dl_dw, in theano)?

it turns out reallyy easy. use torch.autograd.grad

example:

import torch import numpy np torch.autograd import grad  x = torch.autograd.variable(torch.from_numpy(np.random.randn(5, 4))) w = torch.autograd.variable(torch.from_numpy(np.random.randn(4, 3)), requires_grad=true) y = torch.autograd.variable(torch.from_numpy(np.random.randn(5, 3))) loss = ((x.mm(w) - y)**2).sum() (d_loss_d_w, ) = grad(loss, w)  assert np.allclose(d_loss_d_w.data.numpy(), (x.transpose(0, 1).mm(x.mm(w)-y)*2).data.numpy()) 

thanks jerrylin answering question here.