Aloygrad
- Period
- Jan 2026
- Status
- Experiment
- Role
- Engineer
Minimal reverse-mode autodiff engine written from first principles to understand how gradients and neural networks really work.
Minimal reverse-mode automatic differentiation engine built to understand differentiable programming and neural networks from first principles, giving engineers full control over how gradients work.
Automatic differentiation is often learned through mature frameworks that conceal the graph traversal and gradient propagation doing the real work.
Aloygrad rebuilds reverse-mode automatic differentiation from first principles so every operation, dependency, and gradient update remains visible.
How it works
Compose
Build an expression from differentiable values.
Record
Capture operations and dependencies in a graph.
Traverse
Order the graph from output back to inputs.
Differentiate
Apply local derivatives and accumulate gradients.
Key decisions
Prioritize understanding
The implementation stays intentionally small enough to trace instead of competing with production tensor libraries.
Represent the computation graph
Values retain their dependencies so backpropagation can walk the graph in the correct order.
Build from scalar operations
Small mathematical primitives make the chain rule and gradient accumulation concrete.
Technologies
Python / Calculus / Linear Algebra