circuits-llm

Experiments in training and inference for small transformer-style language models, using the circuits ecosystem and hmatrix for linear algebra. Includes byte-pair encoding, attention, GPT-2-shaped forward/backward passes, and AdamW training loops.