| Safe Haskell | None |
|---|---|
| Language | GHC2024 |
NetCoalgebra
Description
Skip-Para neural network expressed as a polynomial coalgebra over the circuits ecosystem.
- Linear maps use
Dense. - Gradients are produced via
DiffP(reverse mode). - Parameter updates run through
sgdProgress. - The whole training step is packaged as a
Coalgebrawhose output direction is a learning-rate scalar.
A These-based boundary type is provided for future batch scheduling;
batching via Tensor is left as a TODO stub.
Synopsis
- data NetParams a = NetParams {}
- netParamsFromArrays :: Array a -> Array a -> Array a -> Array a -> NetParams a
- forward :: (Ord a, Num a, Additive a, Multiplicative a) => NetParams a -> Array a -> Array a
- mseLoss :: (Fractional a, Additive a) => Array a -> Array a -> (a, Array a)
- newtype NetState = NetState {}
- initNetState :: NetParams Double -> NetState
- trainStep :: NetState -> (Array Double, Array Double) -> (Array Double, Double -> NetState)
- netCoalgebra :: Coalgebra NetState (Mono () (Array Double, Array Double)) (Mono Double (Array Double))
- trainNetCoalgebra :: NetParams Double -> Array Double -> Array Double -> Double -> Int -> [(Double, NetParams Double)]
- trainIdentity :: Int -> Double -> NetParams Double -> Array Double -> Array Double -> [(Double, NetParams Double)]
- gradientsViaAD :: NetParams Double -> Array Double -> Array Double -> NetParams Double
- referenceGradients :: NetParams Double -> Array Double -> Array Double -> NetParams Double
- type Boundary a = These a a
- batchSchedule :: a
Parameter bundle (re-exported from Net)
Network parameters. Weights are stored as dense matrices so that
Dense can be used for the linear maps; biases and
activations remain as Array vectors.
Forward pass
forward :: (Ord a, Num a, Additive a, Multiplicative a) => NetParams a -> Array a -> Array a Source #
Run the model with explicit parameters.
mseLoss :: (Fractional a, Additive a) => Array a -> Array a -> (a, Array a) Source #
Mean-squared-error loss and its gradient w.r.t. the prediction.
L = (1/n) Σ (y - target)², dLdy = (2n)(y - target).
Training state
Training state: the parameter bundle. The optimiser is stateless
plain SGD, represented as a Progress that is
instantiated at step time with the current gradient and learning rate.
Training step and coalgebra
trainStep :: NetState -> (Array Double, Array Double) -> (Array Double, Double -> NetState) Source #
One training step.
Returns the prediction and a function that consumes the learning-rate direction and produces the next state.
netCoalgebra :: Coalgebra NetState (Mono () (Array Double, Array Double)) (Mono Double (Array Double)) Source #
Polynomial coalgebra view of the network.
- State:
NetState. - Input position:
(input, target); no input direction. - Output position: prediction; output direction: learning rate.
trainNetCoalgebra :: NetParams Double -> Array Double -> Array Double -> Double -> Int -> [(Double, NetParams Double)] Source #
Train the coalgebra for n steps on a fixed (x, target) pair.
trainIdentity :: Int -> Double -> NetParams Double -> Array Double -> Array Double -> [(Double, NetParams Double)] Source #
Identity-mapping training oracle: train for n steps and return the
loss before each update together with the parameters after each update.
Gradient oracle helpers
gradientsViaAD :: NetParams Double -> Array Double -> Array Double -> NetParams Double Source #
Compute parameter gradients via the 'circuits-ad' DiffP model.
Boundary type
type Boundary a = These a a Source #
A These-based boundary distinguishes inference-only, gradient-only,
and combined inference-with-gradient traffic at a layer boundary.
Batch scheduling stub
batchSchedule :: a Source #
Batch scheduling stub. The intended implementation uses
Tensor combinators (broadcast telecast indexWindows) to
turn single-example layers into batch layers and fold gradients across
the batch dimension.