AI Systems From Scratch - Beginner to OSS Contributor¶
From "I've heard of AI / LLMs" to "I can train a small model, fine-tune a transformer, build a small RAG app, evaluate it honestly, and submit a fix to an AI-adjacent OSS project."
Who this is for¶
- You've finished Python From Scratch (or you're comfortable enough in Python to write small programs).
- You've never trained a model, OR you've copy-pasted some PyTorch / Hugging Face code without really understanding what it does.
Soft prerequisite¶
Python comfort is mandatory - AI tooling lives in Python. If you can't write a function, walk a list, and read a stack trace, do Python From Scratch first.
You do not need a PhD in math. We use linear algebra at the level of "dot products and matrices"; we explain everything else as it appears.
What you'll need¶
- A computer. A GPU helps a lot but isn't required for the first 8 pages. Free options for hands-on GPU work: Google Colab (free tier), Kaggle Notebooks, Lambda Labs (cheap), AWS / GCP (paid).
- Python ≥3.10 (you set this up in the Python beginner path).
- A text editor.
- About 5 hours/week. Path is sized for 4-6 months.
Why AI systems¶
- Biggest growth area in software. The job market and OSS activity around LLMs / ML infra is the most active it's ever been.
- OSS is the heart of the field. PyTorch, Hugging Face, vLLM, llama.cpp, Ollama, LangChain - all open-source and welcoming.
- The barrier is lower than it looks. Modern tooling lets you fine-tune real models with ~50 lines of code; serve them with another ~30.
How this path works¶
Same template as the other beginner paths: one concept per page, code first then walkthrough, exercise, Q&A, done recap.
We use PyTorch as the framework throughout - most popular, best ecosystem. Hugging Face Transformers for pre-trained models. vLLM / Ollama / llama.cpp for inference.
The pages¶
| # | Title | What you'll know after |
|---|---|---|
| 00 | Introduction | What we're doing and why |
| 01 | Setup | Python + PyTorch + CUDA (or CPU) working |
| 02 | Tensors | PyTorch's central data type |
| 03 | Linear algebra you actually need | Dot products, matmul, gradients (intuitive) |
| 04 | Your first neural network | A small MLP from scratch |
| 05 | Training loop | Loss, optimizer, gradient descent |
| 06 | Inference and saving | Loading a pretrained model, running it |
| 07 | Transformers and tokenization | What an LLM actually does |
| 08 | Hugging Face Transformers | Pre-trained models in 3 lines |
| 09 | Fine-tuning | Adapt a model to your data (LoRA-friendly) |
| 10 | Retrieval-Augmented Generation | Embeddings + vector DB + LLM |
| 11 | Evaluation | The hardest part of ML done seriously |
| 12 | Serving models | vLLM, Ollama, simple HTTP wrappers |
| 13 | Picking a project | AI-OSS candidates |
| 14 | Anatomy of an AI OSS project | Case study |
| 15 | Your first contribution | Workflow + PR |
Start with Introduction.
Print this path
Want to read offline or archive? Open the printable version - every section of this path concatenated into one page, styled for paper. Use your browser's Print → Save as PDF.