circuits-diff
Safe HaskellNone
LanguageGHC2024

Circuit.Diff.Circuit

Description

Diff as a circuits arrow.

This module gives the instances that turn the differentiable carrier Diff into a Category with tracing, channels, strength, tensor products, and bimonoid structure. Keeping the instances in the same package as the Diff type avoids the orphan instances that would arise if 'circuits-ad' defined them for a carrier living elsewhere.

Synopsis

Re-exports from the carrier

newtype Diff (p :: k) a b Source #

A reverse-mode differentiable function tagged by a phantom type p.

The phantom tag prevents perturbation confusion: values of type Diff p a b can only be composed with other Diff p values. Nested AD introduces a fresh tag for each level.

runDiff f a returns a pair (b, pullback) where b = f a and pullback maps a cotangent db on the output to a cotangent da on the input.

Constructors

Diff 

Fields

  • runDiff :: a -> (b, b -> a)

    Run the forward pass and return the backward pullback.

Instances

Instances details
Channel Either (Diff p :: Type -> Type -> Type) Source #

Cocartesian channel plumbing for 'Diff.

Instance details

Defined in Circuit.Diff.Circuit

Methods

assoc :: Diff p (Either (Either a b) c) (Either a (Either b c)) #

assoc' :: Diff p (Either a (Either b c)) (Either (Either a b) c) #

slide :: Diff p (Either a (Either b c)) (Either b (Either a c)) #

Channel (,) (Diff p :: Type -> Type -> Type) Source #

Cartesian channel plumbing for 'Diff.

Instance details

Defined in Circuit.Diff.Circuit

Methods

assoc :: Diff p ((a, b), c) (a, (b, c)) #

assoc' :: Diff p (a, (b, c)) ((a, b), c) #

slide :: Diff p (a, (b, c)) (b, (a, c)) #

Strength Either (Diff p :: Type -> Type -> Type) Source #

Cocartesian tensorial strength for 'Diff.

Instance details

Defined in Circuit.Diff.Circuit

Methods

strength :: Diff p b c -> Diff p (Either a b) (Either a c) #

Strength (,) (Diff p :: Type -> Type -> Type) Source #

Cartesian tensorial strength for 'Diff.

Instance details

Defined in Circuit.Diff.Circuit

Methods

strength :: Diff p b c -> Diff p (a, b) (a, c) #

Traced Either (Diff p :: Type -> Type -> Type) Source #

Trace for 'Diff with the Either tensor.

The Either trace is a while-loop: 'Left a' means "iterate again", 'Right c' means "return". Forward pass runs until the body produces a Right, recording each body's pullback. Backward pass replays those pullbacks in reverse order, propagating the output cotangent back through the iteration chain.

The number of iterations is treated as locally constant by the derivative: small perturbations of the input do not change the branch sequence. This is the standard reverse-mode treatment of data-dependent control flow.

Proof obligation (joins the linearity obligation on the other traces): a cotangent on a sum is represented as the same sum, and its tag must match the primal trajectory — the cotangent space at a point of Either a c is the cotangent space of the branch the point is in. Every honest pullback maps an output-tagged cotangent to an input-tagged one; the replay errors loudly on any mismatch rather than misreading a dishonest primitive.

Instance details

Defined in Circuit.Diff.Circuit

Methods

trace :: Diff p (Either a b) (Either a c) -> Diff p b c #

Traced (,) (Diff p :: Type -> Type -> Type) Source #

Trace for 'Diff with the (,) tensor.

The forward pass ties the standard lazy knot:

let (a, c) = body (a, b) in c

The backward pass ties the same shape of knot, transposed. Given dc on the output, the full backward pair (da, db) satisfies a self-referential equation solved by a single lazy binding:

let bd = backward (fst bd, dc) in snd bd

The knot flows through the pair rather than through the channel cotangent alone — backward is called once, the pair is destructured once, and the shape mirrors the forward knot identically.

pullback closes over backward, which closes over the forward pass's intermediates. The closure is the tape: no explicit Wengert list is built because GHC's heap holds the graph. For linear backward maps this is a Neumann series computed lazily; for general maps it is the implicit function theorem as a lazy knot.

Instance details

Defined in Circuit.Diff.Circuit

Methods

trace :: Diff p (a, b) (a, c) -> Diff p b c #

Action (,) (Diff p :: Type -> Type -> Type) Source # 
Instance details

Defined in Circuit.Diff.Circuit

Methods

braid :: Diff p (a, b) (b, a) #

Tensor (,) (Diff p :: Type -> Type -> Type) Source # 
Instance details

Defined in Circuit.Diff.Circuit

Methods

tensor :: Diff p a b -> Diff p c d -> Diff p (a, c) (b, d) #

Unital (,) (Diff p :: Type -> Type -> Type) Source #

Monoidal product for Diff: independent wires, no additive constraint.

>>> let f = Diff (\x -> (x + 1, \d -> d)) :: Diff' Int Int
>>> let g = Diff (\x -> (x * 2, \d -> 2 * d)) :: Diff' Int Int
>>> let (y, pb) = runDiff (tensor f g) (3, 4)
>>> y
(4,8)
>>> pb (1, 1)
(1,2)
Instance details

Defined in Circuit.Diff.Circuit

Methods

unitl :: Diff p (Unit (,), a) a #

unitl' :: Diff p a (Unit (,), a) #

unitr :: Diff p (a, Unit (,)) a #

unitr' :: Diff p a (a, Unit (,)) #

Category (Diff p :: Type -> Type -> Type) Source #

Category for Diff.

'Circuit.Diff still provides Category; circuits needs the local Category with associated Ob (default ()) so Monoidal Traced free Trace folds typecheck after kind-gen.

Instance details

Defined in Circuit.Diff.Circuit

Methods

id :: Diff p a a #

(.) :: Diff p b c -> Diff p a b -> Diff p a c #

Category (Diff p :: Type -> Type -> Type) Source # 
Instance details

Defined in Circuit.Diff

Methods

id :: Diff p a a #

(.) :: Diff p b c -> Diff p a b -> Diff p a c #

Zero (->) a => Discard (Diff p :: Type -> Type -> Type) (a :: Type) Source # 
Instance details

Defined in Circuit.Diff.Circuit

Methods

discard :: Diff p a () #

Zero (->) a => Zero (Diff p :: Type -> Type -> Type) (a :: Type) Source # 
Instance details

Defined in Circuit.Diff.Circuit

Methods

zero :: Diff p () a #

Merge (->) a => Copy (Diff p) a Source #

Copy in D: the pullback is plus (fan-in on the backward pass).

>>> import Circuit.Tensor (Action(..))
>>> import Circuit.Bimonoid (Copy(..), Merge(..))
>>> let (_, pb) = runDiff (copy :: Diff' Int (Int, Int)) 5
>>> pb (1, 2)
3
Instance details

Defined in Circuit.Diff.Circuit

Methods

copy :: Diff p a (a, a) #

Merge (->) a => Merge (Diff p) a Source #

Add in D: the pullback is copy (fan-out on the backward pass).

>>> let (_, pb) = runDiff (plus :: Diff' (Int, Int) Int) (3, 4)
>>> pb 1
(1,1)
Instance details

Defined in Circuit.Diff.Circuit

Methods

plus :: Diff p (a, a) a #

Lit (Diff p Double Double) Source # 
Instance details

Defined in Circuit.Diff.Carrier

(Num s, Num b) => Num (Diff p s b) Source #

Mechanical Num mirror so that base-polymorphic code can also use 'Diff as its carrier. Non-smooth methods (abs, signum) raise an error; the useful cases (literals, +, *, -) work out of the box.

Instance details

Defined in Circuit.Diff

Methods

(+) :: Diff p s b -> Diff p s b -> Diff p s b #

(-) :: Diff p s b -> Diff p s b -> Diff p s b #

(*) :: Diff p s b -> Diff p s b -> Diff p s b #

negate :: Diff p s b -> Diff p s b #

abs :: Diff p s b -> Diff p s b #

signum :: Diff p s b -> Diff p s b #

fromInteger :: Integer -> Diff p s b #

(Additive s, Additive b) => Additive (Diff p s b) Source #

Additive structure: sum rule.

zero is the constant zero function; (+) adds outputs and fans-in cotangents.

Instance details

Defined in Circuit.Diff

Methods

(+) :: Diff p s b -> Diff p s b -> Diff p s b #

zero :: Diff p s b #

(Additive s, Subtractive s, Subtractive b) => Subtractive (Diff p s b) Source #

Subtractive structure: negation pushes through the pullback.

Instance details

Defined in Circuit.Diff

Methods

negate :: Diff p s b -> Diff p s b #

(-) :: Diff p s b -> Diff p s b -> Diff p s b #

(ExpField b, Additive s, Subtractive s, Multiplicative b) => ExpField (Diff p s b) Source #

Exponential field: exp, log, and the derived power/root family.

Instance details

Defined in Circuit.Diff

Methods

exp :: Diff p s b -> Diff p s b #

log :: Diff p s b -> Diff p s b #

(**) :: Diff p s b -> Diff p s b -> Diff p s b #

logBase :: Diff p s b -> Diff p s b -> Diff p s b #

sqrt :: Diff p s b -> Diff p s b #

(TrigField b, ExpField b, Additive s, Subtractive s, Multiplicative b, Divisive b) => TrigField (Diff p s b) Source #

Trigonometric field: the elementary transcendental family.

Instance details

Defined in Circuit.Diff

Methods

pi :: Diff p s b #

sin :: Diff p s b -> Diff p s b #

cos :: Diff p s b -> Diff p s b #

tan :: Diff p s b -> Diff p s b #

asin :: Diff p s b -> Diff p s b #

acos :: Diff p s b -> Diff p s b #

atan :: Diff p s b -> Diff p s b #

atan2 :: Diff p s b -> Diff p s b -> Diff p s b #

sinh :: Diff p s b -> Diff p s b #

cosh :: Diff p s b -> Diff p s b #

tanh :: Diff p s b -> Diff p s b #

asinh :: Diff p s b -> Diff p s b #

acosh :: Diff p s b -> Diff p s b #

atanh :: Diff p s b -> Diff p s b #

(Additive s, Subtractive b, Multiplicative b, Divisive b) => Divisive (Diff p s b) Source #

Divisive structure: reciprocal rule.

Division inherits the product rule via the default / = * . recip.

Instance details

Defined in Circuit.Diff

Methods

recip :: Diff p s b -> Diff p s b #

(/) :: Diff p s b -> Diff p s b -> Diff p s b #

(Additive s, Multiplicative b) => Multiplicative (Diff p s b) Source #

Multiplicative structure: product rule.

one is the constant one function.

Instance details

Defined in Circuit.Diff

Methods

(*) :: Diff p s b -> Diff p s b -> Diff p s b #

one :: Diff p s b #

type Diff' = Diff () Source #

The untagged differentiable arrow. Existing code can continue to use this; it is simply Diff ().

Traced variants

traceNFrom :: forall {k} a (p :: k) b c. MergeZero (->) a => a -> Int -> Diff p (a, b) (a, c) -> Diff p b c Source #

Iterated trace for strict carriers.

The lazy trace diverges on strict cotangent types (Double, etc.) when the feedback channel has nonzero self-coupling (∂a_out/∂a_in ≠ 0). traceNFrom replaces the lazy knot with truncated fixed-point iteration.

  • Forward — iterate from caller-supplied seed x0, N steps. There is no canonical seed for the forward pass (the fixpoint is arbitrary nonlinear), so the caller provides one.
  • Backward — iterate from zero, N steps, extract snd once. The backward equation is guaranteed affine (calculus promises linearity in cotangents), so zero is the principled seed. The Neumann summation happens inside the iteration — no double-counting, and plus retreats to where the theory says it lives: inside prims and Copy.

Lives beside the lawful-but-lazy instance, not replacing it.

traceStarFrom Source #

Arguments

:: forall {k} j c (p :: k) b. (StarSemiring j, MergeZero (->) c) 
=> j

forward seed

-> Int

forward iteration count

-> Diff p (j, b) (j, c) 
-> Diff p b c 

Trace with a closed-form backward pass via the Kleene star.

The integer n in traceNFrom truncates a series on both passes. But only the forward fixpoint is genuinely nonlinear; the backward channel equation is affine — calculus promises linearity in cotangents:

da = A·da + C·dc        solution:  da = star A · C·dc

with star a = one + a·star a — the Neumann series as algebra, 1/(1−a) over a field. Because the pullback is linear, the blocks A and C·dc are extractable by probing:

backward (dj, dc) = (A·dj + C·dc, B·dj + D·dc)
A    = fst (backward (one,  0))   -- channel self-coupling
C·dc = fst (backward (zero, dc))

and the trace's pullback is the Schur complement D·dc + B·star A·C·dc, recovered with one more probe at the backward fixpoint:

db = snd (backward (star A · C·dc, dc))

(Check: A·(star A·C·dc) + C·dc = (A·star A + one)·C·dc = star A·C·dc — the star law discharges the fixpoint.)

So: forward still iterates from the caller's seed (no closed form exists for an arbitrary nonlinear fixpoint), but the backward pass is exact in three calls to backward — no Neumann index at all. The star probe is computed once per forward point and shared across all cotangents.

The closed form is the truncated iteration's limit; pure-Prelude witness at a = 0.3, c = 2:

>>> let daIter = iterate (\d -> 0.3 * d + 2.0 * 1.0) 0 !! 200
>>> abs (daIter - 1.0 / (1.0 - 0.3) * 2.0) < 1e-12
True

Caveat: numhask declares StarSemiring but ships no instances; the only carriers in the tower are FieldStar (star a = recip (1−a)), Warshall, and MinPlus. For bare Double channels and for vector channels solved by starMatrix, see Circuit.Diff.Star — the Schur-complement bridge proper.

Proof obligation: the probes assume the pullback is linear. Every honestly-constructed 'Diff primitive satisfies this (a pullback is a linear map); a primitive whose backward closure is affine-with-offset is a bug that this function will silently misread.

traceStar :: forall {k} (p :: k) j b c. Diff p (j, b) (j, c) -> Diff p b c Source #

Trace via the Kleene star — the execution formula, lazy form.

For a knot body with channel self-coupling block A and cross-blocks B, C, D, the trace is the Schur complement:

traceStar f = D + B · star A · C

The lazy trace instance for 'Diff computes exactly this via a lazy fixpoint rather than closed form, so this alias is definable without using star at all. Note that numhask ships no StarSemiring instances, so for concrete carriers prefer traceStarFrom (scalar channel, closed-form backward) or Circuit.Diff.Star.traceStarMatrix (vector channel, solved by starMatrix — the bridge made literal).

Smoke test

quadD :: forall {k} (p :: k). Diff p Double Double Source #

2x² + 3x + 5 built from tensor, dup, and plus on the Diff arrow. No Net needed — the instances are the denotations the rows will realise to.

The gradient is 4x + 3, so at x = 1: value 10, gradient 7.

>>> let (y, pb) = runDiff quadD 1.0
>>> y
10.0
>>> pb 1.0
7.0

Orphan instances

Channel Either (Diff p :: Type -> Type -> Type) Source #

Cocartesian channel plumbing for 'Diff.

Instance details

Methods

assoc :: Diff p (Either (Either a b) c) (Either a (Either b c)) #

assoc' :: Diff p (Either a (Either b c)) (Either (Either a b) c) #

slide :: Diff p (Either a (Either b c)) (Either b (Either a c)) #

Channel (,) (Diff p :: Type -> Type -> Type) Source #

Cartesian channel plumbing for 'Diff.

Instance details

Methods

assoc :: Diff p ((a, b), c) (a, (b, c)) #

assoc' :: Diff p (a, (b, c)) ((a, b), c) #

slide :: Diff p (a, (b, c)) (b, (a, c)) #

Strength Either (Diff p :: Type -> Type -> Type) Source #

Cocartesian tensorial strength for 'Diff.

Instance details

Methods

strength :: Diff p b c -> Diff p (Either a b) (Either a c) #

Strength (,) (Diff p :: Type -> Type -> Type) Source #

Cartesian tensorial strength for 'Diff.

Instance details

Methods

strength :: Diff p b c -> Diff p (a, b) (a, c) #

Traced Either (Diff p :: Type -> Type -> Type) Source #

Trace for 'Diff with the Either tensor.

The Either trace is a while-loop: 'Left a' means "iterate again", 'Right c' means "return". Forward pass runs until the body produces a Right, recording each body's pullback. Backward pass replays those pullbacks in reverse order, propagating the output cotangent back through the iteration chain.

The number of iterations is treated as locally constant by the derivative: small perturbations of the input do not change the branch sequence. This is the standard reverse-mode treatment of data-dependent control flow.

Proof obligation (joins the linearity obligation on the other traces): a cotangent on a sum is represented as the same sum, and its tag must match the primal trajectory — the cotangent space at a point of Either a c is the cotangent space of the branch the point is in. Every honest pullback maps an output-tagged cotangent to an input-tagged one; the replay errors loudly on any mismatch rather than misreading a dishonest primitive.

Instance details

Methods

trace :: Diff p (Either a b) (Either a c) -> Diff p b c #

Traced (,) (Diff p :: Type -> Type -> Type) Source #

Trace for 'Diff with the (,) tensor.

The forward pass ties the standard lazy knot:

let (a, c) = body (a, b) in c

The backward pass ties the same shape of knot, transposed. Given dc on the output, the full backward pair (da, db) satisfies a self-referential equation solved by a single lazy binding:

let bd = backward (fst bd, dc) in snd bd

The knot flows through the pair rather than through the channel cotangent alone — backward is called once, the pair is destructured once, and the shape mirrors the forward knot identically.

pullback closes over backward, which closes over the forward pass's intermediates. The closure is the tape: no explicit Wengert list is built because GHC's heap holds the graph. For linear backward maps this is a Neumann series computed lazily; for general maps it is the implicit function theorem as a lazy knot.

Instance details

Methods

trace :: Diff p (a, b) (a, c) -> Diff p b c #

Action (,) (Diff p :: Type -> Type -> Type) Source # 
Instance details

Methods

braid :: Diff p (a, b) (b, a) #

Tensor (,) (Diff p :: Type -> Type -> Type) Source # 
Instance details

Methods

tensor :: Diff p a b -> Diff p c d -> Diff p (a, c) (b, d) #

Unital (,) (Diff p :: Type -> Type -> Type) Source #

Monoidal product for Diff: independent wires, no additive constraint.

>>> let f = Diff (\x -> (x + 1, \d -> d)) :: Diff' Int Int
>>> let g = Diff (\x -> (x * 2, \d -> 2 * d)) :: Diff' Int Int
>>> let (y, pb) = runDiff (tensor f g) (3, 4)
>>> y
(4,8)
>>> pb (1, 1)
(1,2)
Instance details

Methods

unitl :: Diff p (Unit (,), a) a #

unitl' :: Diff p a (Unit (,), a) #

unitr :: Diff p (a, Unit (,)) a #

unitr' :: Diff p a (a, Unit (,)) #

Category (Diff p :: Type -> Type -> Type) Source #

Category for Diff.

'Circuit.Diff still provides Category; circuits needs the local Category with associated Ob (default ()) so Monoidal Traced free Trace folds typecheck after kind-gen.

Instance details

Methods

id :: Diff p a a #

(.) :: Diff p b c -> Diff p a b -> Diff p a c #

Zero (->) a => Discard (Diff p :: Type -> Type -> Type) (a :: Type) Source # 
Instance details

Methods

discard :: Diff p a () #

Zero (->) a => Zero (Diff p :: Type -> Type -> Type) (a :: Type) Source # 
Instance details

Methods

zero :: Diff p () a #

Merge (->) a => Copy (Diff p) a Source #

Copy in D: the pullback is plus (fan-in on the backward pass).

>>> import Circuit.Tensor (Action(..))
>>> import Circuit.Bimonoid (Copy(..), Merge(..))
>>> let (_, pb) = runDiff (copy :: Diff' Int (Int, Int)) 5
>>> pb (1, 2)
3
Instance details

Methods

copy :: Diff p a (a, a) #

Merge (->) a => Merge (Diff p) a Source #

Add in D: the pullback is copy (fan-out on the backward pass).

>>> let (_, pb) = runDiff (plus :: Diff' (Int, Int) Int) (3, 4)
>>> pb 1
(1,1)
Instance details

Methods

plus :: Diff p (a, a) a #