Similarly to @nograd in Zygote, we need a @non_differentiable macro to make it straightforward for someone to declare that the particular method of a function should have an frule and rrule that return zeros / related.
Proposed Usage:
@non_differentiable f(x::Tx, y...)
should be expanded to something like
function frule((::Any, ::Any, ::Vararg), ::typeof(f), x::Tx, y...)
return f(x, y...), Zero()
end
function rrule(::typeof(f), x::Tx, y...)
function pullback_f_non_differentiable(::Any)
return (DoesNotExist(), DoesNotExist(), map(DoesNotExist(), 1:length(y)))
end
return f(x, y...), pullback_f_non_differentiable
endI think the frule and rrule that something should expand to once we've established the API is pretty clear (I'm not sure I've actually got the rrule correct for the vararg in the above). Same thing for what the kwargs bit of the API should look like.
So the point of discussion is does the above form of @non_differentiable do what we want it to do? In particular, do we want to insist / not insist that people provide argument names?
Similarly to
@nogradin Zygote, we need a@non_differentiablemacro to make it straightforward for someone to declare that the particular method of a function should have anfruleandrrulethat return zeros / related.Proposed Usage:
should be expanded to something like
I think the
fruleandrrulethat something should expand to once we've established the API is pretty clear (I'm not sure I've actually got therrulecorrect for the vararg in the above). Same thing for what thekwargsbit of the API should look like.So the point of discussion is does the above form of
@non_differentiabledo what we want it to do? In particular, do we want to insist / not insist that people provide argument names?