you can build your own small language model from scratch on your computer - this free app walks you through every step 🤯
it's called Language Model Builder, and it comes in two parts.
the first is a 90-minute interactive textbook.
it covers the fundamentals, tokenization, embeddings, attention, the transformer, training data, loss and gradient descent, fine-tuning. no coding or math background needed.
and instead of staring at diagrams, you run each experiment yourself as you read.
the second is a training workbench that runs the full pipeline locally on Apple's MLX:
> pre-train a base model on a dataset you pick
> fine-tune it with SFT, then align it with DPO
> watch the loss curve drop in real time as it learns
> chat with the model you made, with a token-probability x-ray view so you can see how it "thinks"
on default settings you'll get coherent, multi-paragraph text in about a day.
a GPT-2-small-class model takes roughly a week on a high-end Mac.
100% free, no account, runs entirely offline.
Video