[rxwei]

This PR removes the CotangentVector associated type and make it equal to TangentVector.

Mathematically, types conforming to the Differentiable protocol represent a Riemannian manifold, whose metric is inner products, which provide an isomorphism between a tangent space and a cotangent space at a point.

Some theoretical and practical reasons:

• It is not possible to correctly represent dual vectors in Swift today because functions cannot conform to protocols ((TangentVector) -> Scalar cannot conform to AdditiveArithmetic).
• In computation, it is always more practical to compute numerical tangent vectors instead of true cotangent vectors (partially applied dot product functions).
• The mutually recursive generic constraints triggered a significant bug in the type checker (SR-9595) that required a lot of workarounds (TF-213), in particular, an ugly split of 3 protocols: __Differentiable, _Differentiable, and Differentiable.
• The split between TangentVector and CotangentVector makes user-defined conformances complicated.

Changes include:

• Remove associatedtype CotangentVector from Differentiable.
• Define typealias CotangentVector = TangentVector in Differentiable with a deprecation message.
• Merge __Differentiable and _Differentiable back into Differentiable.
• Replace all mentions of "cotangent" with "tangent" in the code base.
• Adapt derived conformances logic.
• Adapt tests.

Defines away TF-213.

[rxwei]

@swift-ci please test tensorflow

[dan-zheng]

Exciting simplification!
I skimmed the files, LGTM if tests pass.

[rxwei]

@swift-ci please clean test tensorflow Linux

[rxwei]

All tests pass locally, but CI seems dead.

[dan-zheng]

How about alternative names for TangentVector, like Derivative?

I feelTangentVector may be more mathematically correct and obscure, while names like Derivative are less correct but more familiar.

cc @marcrasi

[marcrasi]

I like the idea of changing it to something more familiar. Ideas:

• DerivativeVector (I think Derivative is too easy to confuse with the actual differential/pullback/gradient functions, but sticking Vector after it clarifies that it's a single value.)
• GradientVector
• ChangeVector
• RateOfChangeVector
• Velocity (This one is nice because it's really concrete and familiar. But it's only technically correct when you're taking a derivative of a path in space with respect to time. And it would be super confusing if you take a second derivative with respect to time, and get an acceleration value with type Velocity).

[saeta]

+1 to DerivativeVector :-)