circuits-diff
Safe HaskellNone
LanguageGHC2024

Circuit.Diff.Metric

Description

Metric-aware transpose for differentiable arrows.

A metric g turns the bare dagger (transpose) of a 'Diff into the true adjoint g_a^-1 . J^T . g_b. The same combinator also supports optimizer-style preconditioning, where g is a diagonal metric diag(sqrt v + eps) and only the lowering half is used.

Synopsis

Adjoint with respect to domain/codomain metrics

adjointWith Source #

Arguments

:: Diff' (a, a) a

g_a^-1 — raise a covector on the domain to a vector

-> Diff' (b, b) b

g_b — lower a vector on the codomain to a covector

-> Diff' a b
J : a -> b
-> Diff' a b

J with its pullback conjugated to the g-adjoint. The type is still Diff' a b because the lens stores the forward map a -> b and the backward map b -> a; the adjoint lives in the pullback.

Adjoint of J : a -> b with respect to domain metric g_a and codomain metric g_b.

The metrics are themselves 'Diffs of type Diff' (point, vector) vector: the forward pass lowers or raises a vector at the given point. The backward pass of the metric carries ∂g in its point-slot; adjointWith uses only the forward passes.

Forward-only metric application

raiseWith :: Diff' (s, a) a -> s -> a -> a Source #

Raise a covector using a metric at the given point.

lowerWith :: Diff' (s, a) a -> s -> a -> a Source #

Lower a vector using a metric at the given point.

Common metrics

diagonalMetric Source #

Arguments

:: Additive s 
=> (s -> a -> a)

f such that g(s) = diag(f s)

-> Diff' (s, a) a 

Diagonal metric g(s) = diag(f s).

Lowering maps x -> f s * x; raising maps c -> recip (f s) * c. The pullback's state-slot is zero because the typical consumer (an optimizer) does not back-propagate through the metric coefficients. If you need Christoffel-style ∂g, write a custom metric 'Diff.

euclideanMetric :: Additive a => Diff' (a, a) a Source #

Euclidean metric g = δ: raise and lower are both the identity.