| Safe Haskell | None |
|---|---|
| Language | GHC2024 |
Circuit.Diff.Array
Description
Array AD on top of Array and Square.
This module provides:
- forward-mode Taylor towers for square-matrix computations, via
Jetinstantiated at 'Square n Double'; - elementwise towers by mapping scalar
Jets across array elements; - explicit reverse-mode
Diffprimitives for matrix multiplication, transpose, sum, scale, and elementwise activations.
The generic Multiplicative instance of Diff is not used for matrix
multiplication because its product-rule pullback is the wrong adjoint for
non-commutative multiplication.
Synopsis
- tensorTaylor :: forall (n :: Nat) a. (KnownNat n, Additive a, Multiplicative a, FromInteger a) => (Jet (Square n a) -> Jet (Square n a)) -> Int -> Square n a -> [Square n a]
- tensorDerivativeN :: forall (n :: Nat) a. (KnownNat n, Additive a, Multiplicative a, FromInteger a) => (Jet (Square n a) -> Jet (Square n a)) -> Int -> Square n a -> Square n a
- elementwiseTower :: forall (s :: [Nat]) a. (KnownNats s, Additive a, Multiplicative a, FromInteger a) => (Jet a -> Jet a) -> Int -> Array s a -> Array s a
- matMulD :: forall {k} (n :: Nat) (p :: k). KnownNat n => Diff p (Square n Double, Square n Double) (Square n Double)
- transposeD :: forall {k} (m :: Nat) (n :: Nat) (p :: k). (KnownNat m, KnownNat n) => Diff p (Array '[m, n] Double) (Array '[n, m] Double)
- sumD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => Diff p (Array s Double) Double
- scaleD :: forall {k} (s :: [Nat]) (p :: k). Double -> Diff p (Array s Double) (Array s Double)
- elementwiseD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => (Double -> (Double, Double -> Double)) -> Diff p (Array s Double) (Array s Double)
- sigmoidD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => Diff p (Array s Double) (Array s Double)
- tanhD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => Diff p (Array s Double) (Array s Double)
Forward-mode tensor towers
tensorTaylor :: forall (n :: Nat) a. (KnownNat n, Additive a, Multiplicative a, FromInteger a) => (Jet (Square n a) -> Jet (Square n a)) -> Int -> Square n a -> [Square n a] Source #
First k raw derivatives of a square-matrix function at x0.
The function must be built from NumHask-polymorphic operations so that the
Jet recurrence engine can propagate matrix coefficients through matrix
multiplication.
tensorDerivativeN :: forall (n :: Nat) a. (KnownNat n, Additive a, Multiplicative a, FromInteger a) => (Jet (Square n a) -> Jet (Square n a)) -> Int -> Square n a -> Square n a Source #
The n-th raw derivative of a square-matrix function at x0.
elementwiseTower :: forall (s :: [Nat]) a. (KnownNats s, Additive a, Multiplicative a, FromInteger a) => (Jet a -> Jet a) -> Int -> Array s a -> Array s a Source #
Apply a scalar tower function elementwise to every entry of an array.
This is the elementwise analogue of Eshkol's tt-sigmoid / tt-tanh:
each component gets its own scalar Taylor tower, independent of the others.
Reverse-mode tensor primitives
matMulD :: forall {k} (n :: Nat) (p :: k). KnownNat n => Diff p (Square n Double, Square n Double) (Square n Double) Source #
Matrix multiplication with the correct reverse-mode adjoint.
For Y = A B the pullback is dA = dY B^T, dB = A^T dY.
transposeD :: forall {k} (m :: Nat) (n :: Nat) (p :: k). (KnownNat m, KnownNat n) => Diff p (Array '[m, n] Double) (Array '[n, m] Double) Source #
Transpose. Its own adjoint.
sumD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => Diff p (Array s Double) Double Source #
Sum all elements of an array. Pullback replicates the scalar.
scaleD :: forall {k} (s :: [Nat]) (p :: k). Double -> Diff p (Array s Double) (Array s Double) Source #
Scale every element by a constant.
elementwiseD :: forall {k} (s :: [Nat]) (p :: k). KnownNats s => (Double -> (Double, Double -> Double)) -> Diff p (Array s Double) (Array s Double) Source #
Elementwise activation from a scalar primitive x -> (y, dy/dx).