| Safe Haskell | None |
|---|---|
| Language | GHC2024 |
Circuit.Learn.Adam
Contents
Synopsis
- ewma :: Double -> Double -> Process Double Double
- ewmaDirect :: Double -> Double -> [Double] -> [Double]
- adam :: Double -> Double -> Double -> Double -> Process Double Double
- adamDecomposed :: Double -> Double -> Double -> Double -> [Double] -> [Double]
- adamReference :: Double -> Double -> Double -> [Double] -> [(Double, Double, Int)]
- adamUpdates :: Double -> Double -> Double -> Double -> [(Double, Double, Int)] -> [Double]
EWMA building block
ewma :: Double -> Double -> Process Double Double Source #
Exponentially weighted moving average as a Process.
State is the current EWMA value; output is the same value.
ewmaDirect :: Double -> Double -> [Double] -> [Double] Source #
Direct EWMA recurrence on a list (no Process overhead).
First element is after first observation.
Adam
adam :: Double -> Double -> Double -> Double -> Process Double Double Source #
Adam parameter update as a Process.
Input: gradient. Output: parameter update (not the new parameter).
State: (m, v, t) — first moment, second moment, timestep.
adamDecomposed :: Double -> Double -> Double -> Double -> [Double] -> [Double] Source #
Adam decomposed into two independent EWMA channels (the frontier claim).
The m channel is an EWMA of raw gradients (beta1-weighted), the v channel
is an EWMA of squared gradients (beta2-weighted). Each is computed
independently via scanl', then combined with the bias-corrected quotient.
This produces exactly the same updates as the monolithic adam Process.