| Safe Haskell | None |
|---|---|
| Language | GHC2024 |
Circuit.LLM.Inference
Contents
Description
Token sampling and auto-regressive text generation.
Synopsis
- greedySample :: Vector Double -> Int
- temperatureSample :: RandomGen g => Double -> Vector Double -> g -> (Int, g)
- topKSample :: RandomGen g => Int -> Double -> Vector Double -> g -> (Int, g)
- generate :: GptConfig -> Gpt -> BPEModel -> Text -> Int -> IO Text
- argmax :: Vector Double -> Int
- softmaxV :: Vector Double -> Vector Double
- lastRow :: Matrix Double -> Vector Double
Sampling
temperatureSample :: RandomGen g => Double -> Vector Double -> g -> (Int, g) Source #
Temperature sampling.
topKSample :: RandomGen g => Int -> Double -> Vector Double -> g -> (Int, g) Source #
Top-K sampling.
Generation
generate :: GptConfig -> Gpt -> BPEModel -> Text -> Int -> IO Text Source #
Auto-regressive text generation using greedy sampling.