mnet
Safe HaskellNone
LanguageGHC2024

NetCoalgebra

Description

Skip-Para neural network expressed as a polynomial coalgebra over the circuits ecosystem.

  • Linear maps use Dense.
  • Gradients are produced via DiffP (reverse mode).
  • Parameter updates run through sgd Progress.
  • The whole training step is packaged as a Coalgebra whose output direction is a learning-rate scalar.

A These-based boundary type is provided for future batch scheduling; batching via Tensor is left as a TODO stub.

Synopsis

Parameter bundle (re-exported from Net)

data NetParams a Source #

Network parameters. Weights are stored as dense matrices so that Dense can be used for the linear maps; biases and activations remain as Array vectors.

Constructors

NetParams 

Fields

Instances

Instances details
Eq a => Eq (NetParams a) Source # 
Instance details

Defined in Net

Methods

(==) :: NetParams a -> NetParams a -> Bool #

(/=) :: NetParams a -> NetParams a -> Bool #

Show a => Show (NetParams a) Source # 
Instance details

Defined in Net

netParamsFromArrays :: Array a -> Array a -> Array a -> Array a -> NetParams a Source #

Build NetParams from plain Array weights and biases.

Forward pass

forward :: (Ord a, Num a, Additive a, Multiplicative a) => NetParams a -> Array a -> Array a Source #

Run the model with explicit parameters.

mseLoss :: (Fractional a, Additive a) => Array a -> Array a -> (a, Array a) Source #

Mean-squared-error loss and its gradient w.r.t. the prediction.

L = (1/n) Σ (y - target)², dLdy = (2n)(y - target).

Training state

newtype NetState Source #

Training state: the parameter bundle. The optimiser is stateless plain SGD, represented as a Progress that is instantiated at step time with the current gradient and learning rate.

Constructors

NetState 

initNetState :: NetParams Double -> NetState Source #

Initial training state.

Training step and coalgebra

trainStep :: NetState -> (Array Double, Array Double) -> (Array Double, Double -> NetState) Source #

One training step.

Returns the prediction and a function that consumes the learning-rate direction and produces the next state.

netCoalgebra :: Coalgebra NetState (Mono () (Array Double, Array Double)) (Mono Double (Array Double)) Source #

Polynomial coalgebra view of the network.

  • State: NetState.
  • Input position: (input, target); no input direction.
  • Output position: prediction; output direction: learning rate.

trainNetCoalgebra :: NetParams Double -> Array Double -> Array Double -> Double -> Int -> [(Double, NetParams Double)] Source #

Train the coalgebra for n steps on a fixed (x, target) pair.

trainIdentity :: Int -> Double -> NetParams Double -> Array Double -> Array Double -> [(Double, NetParams Double)] Source #

Identity-mapping training oracle: train for n steps and return the loss before each update together with the parameters after each update.

Gradient oracle helpers

gradientsViaAD :: NetParams Double -> Array Double -> Array Double -> NetParams Double Source #

Compute parameter gradients via the 'circuits-ad' DiffP model.

referenceGradients :: NetParams Double -> Array Double -> Array Double -> NetParams Double Source #

Hand-rolled reference gradients for the 2-layer network, kept to verify the 'circuits-ad' / DiffP gradients in the oracle.

This is the old skip-Para chain rule, written directly against the same Dense primitives used by the rest of the module.

Boundary type

type Boundary a = These a a Source #

A These-based boundary distinguishes inference-only, gradient-only, and combined inference-with-gradient traffic at a layer boundary.

  • This prediction — inference only, no gradient.
  • That gradient — training signal only, no prediction.
  • These prediction gradient — training with prediction (common supervised case).

Batch scheduling stub

batchSchedule :: a Source #

Batch scheduling stub. The intended implementation uses Tensor combinators (broadcast telecast indexWindows) to turn single-example layers into batch layers and fold gradients across the batch dimension.