circuits
Safe HaskellNone
LanguageGHC2024

Circuit.Pullback

Description

Linear cotangent maps — the base arrow for reverse-mode gradients.

Pullback b a is a linear map b -> a read as an arrow from b (output cotangent) to a (input cotangent). Composition is plain function composition — the reversal is not in this category, it is in how a net is transposed so that output cotangents flow back to input cotangents. Within an arrow the chain rule is then just (.).

This is the linear semantics behind reverse-mode automatic differentiation: a Net whose wires carry pullbacks rather than smooth maps, avoiding the second-derivative confusion that comes from trying to compose Diff arrows directly.

Synopsis

Linear cotangent arrow

newtype Pullback b a Source #

A linear map from output cotangents to input cotangents, read as an arrow b -> a.

>>> let pb = Pullback (*2) :: Pullback Double Double
>>> runPullback pb 3
6.0

Constructors

Pullback 

Fields

  • runPullback :: b -> a

    Apply the pullback to an output cotangent.

Instances

Instances details
Copy Pullback a Source #

Pullback-instance of the comonoid structure.

Copy's pullback is addition; discard's pullback is the zero cotangent. These are not used by the transposition step of reverse-mode AD (which encodes structural rows as Lifts to avoid channel-type constraints), but they make Pullback a full bimonoid carrier.

>>> runPullback (copy :: Pullback Int (Int, Int)) 3
(3,3)
>>> runPullback (discard :: Pullback Int ()) 5
()

NOTE: neither method here uses an Additive (->) a constraint — copying and discarding are linear as they stand. If the class head permits, drop the constraint; keeping a stray Additive reads as "addition happens in this instance", which is the confusion the paragraph above tries to dispel.

Instance details

Defined in Circuit.Pullback

Methods

copy :: Pullback a (a, a) Source #

Merge (->) a => Merge Pullback a Source #

Pullback-instance of the additive/monoid structure.

Addition's pullback is copying; zero's pullback is discarding.

>>> runPullback (plus :: Pullback (Int, Int) Int) (1, 2)
3
>>> runPullback (zero :: Pullback () Int) ()
0
Instance details

Defined in Circuit.Pullback

Methods

plus :: Pullback (a, a) a Source #

Category Pullback Source # 
Instance details

Defined in Circuit.Pullback

Methods

id :: Pullback a a Source #

(.) :: Pullback b c -> Pullback a b -> Pullback a c Source #

Discard Pullback (a :: Type) Source # 
Instance details

Defined in Circuit.Pullback

Methods

discard :: Pullback a () Source #

Zero (->) a => Zero Pullback (a :: Type) Source # 
Instance details

Defined in Circuit.Pullback

Methods

zero :: Pullback () a Source #

Channel (,) Pullback Source #

Cartesian channel plumbing for pullbacks.

Instance details

Defined in Circuit.Pullback

Methods

assoc :: Pullback ((a, b), c) (a, (b, c)) Source #

assoc' :: Pullback (a, (b, c)) ((a, b), c) Source #

slide :: Pullback (a, (b, c)) (b, (a, c)) Source #

Strength (,) Pullback Source # 
Instance details

Defined in Circuit.Pullback

Methods

strength :: Pullback b c -> Pullback (a, b) (a, c) Source #

Traced (,) Pullback Source #

The cartesian trace for pullbacks.

The body is a linear map f :: (x, c) -> (x, b). The traced pullback c -> b solves the affine feedback equation in cotangent space:

(dx, db) = f (dx, dc)

solved by the same lazy knot that a differentiable arrow uses. For strict carriers with nonzero channel self-coupling this diverges, exactly as the lazy differentiable trace does. Unlike the differentiable case, though, the equation here is always affinePullback arrows are linear by construction — so a knot over a star-semiring carrier can be eliminated outright rather than iterated.

>>> let body = Pullback (\(dx', dc) -> (2.0 * dc, dx')) :: Pullback (Double, Double) (Double, Double)
>>> runPullback (trace body) 1.0
2.0
Instance details

Defined in Circuit.Pullback

Methods

trace :: Pullback (a, b) (a, c) -> Pullback b c Source #

Action (,) Pullback Source # 
Instance details

Defined in Circuit.Pullback

Methods

braid :: Pullback (a, b) (b, a) Source #

Tensor (,) Pullback Source # 
Instance details

Defined in Circuit.Pullback

Methods

tensor :: Pullback a b -> Pullback c d -> Pullback (a, c) (b, d) Source #

Unital (,) Pullback Source #

Parallel composition pairs pullbacks independently; braid swaps the two cotangents.

>>> let f = Pullback (+1) :: Pullback Int Int
>>> let g = Pullback (*2) :: Pullback Int Int
>>> runPullback (tensor f g) (3, 4)
(4,8)
Instance details

Defined in Circuit.Pullback

Running a pullback net

evalPullback :: Net (,) Pullback b a -> b -> a Source #

Evaluate a pullback net at a single output cotangent.

This is the one-shot reverse pass: the net was built by transposing a smooth net, and applying it to a cotangent db yields the input cotangent da.