About the Course
CSC 352/552 Generative AI is an advanced course that takes students from foundational LLM concepts to cutting-edge large-scale AI system implementation. Starting with Raschka's hands-on approach to building language models from scratch, students master tokenization, attention mechanisms, and transformer architectures. Later meetings cover prompting, retrieval-augmented generation (RAG), tool use, and related methods. Students complete hands-on projects implementing core LLM components.
Course Schedule
Course Materials
Available Lectures
| Lecture | Topic | Materials |
|---|---|---|
| -- | Course Introduction and Overview | Syllabus review, course expectations |
| 1 | Machine Learning Foundations | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 2 | PyTorch tensors and autograd | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 3 | PyTorch networks and training | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 4 | Tokenization | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 5 | Embeddings and data sampling | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 6 | Attention and Transformer Basics | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 7 | Causal and multi-head attention | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 8 | Building GPT Architecture | Posted when we reach this meeting |
| 9 | GPT implementation | Posted when we reach this meeting |
| 10 | Model Training Pipeline | Posted when we reach this meeting |
| 11 | Training pipeline practice | Posted when we reach this meeting |
| 12 | Fine-tuning for Classification | Posted when we reach this meeting |
| 13 | Classification fine-tuning practice | Posted when we reach this meeting |
| 14 | Instruction Fine-tuning | Posted when we reach this meeting |
| 15 | Parameter-efficient fine-tuning / LoRA | Posted when we reach this meeting |
| 16 | Agent harness: the agent loop | Posted when we reach this meeting |
| 17 | Tool use and function calling | Posted when we reach this meeting |
| 18 | MCP / tool harness | Posted when we reach this meeting |
| 19 | Agent harness practice | Posted when we reach this meeting |
| 20 | Systematic prompting | Posted when we reach this meeting |
| 21 | Chain-of-Thought and reasoning patterns | Posted when we reach this meeting |
| 22 | Prompting experiments | Posted when we reach this meeting |
| 23 | RAG foundations | Posted when we reach this meeting |
| 24 | RAG pipelines and evaluation | Posted when we reach this meeting |
| 25 | RAG practice | Posted when we reach this meeting |
| 26 | Reasoning models and inference-time compute | Posted when we reach this meeting |
| 27 | Final project kickoff | Posted when we reach this meeting |
Assignments
| Assignment | Topic | Due | Materials |
|---|---|---|---|
| 1 | Lectures 1 to 3 | Sun, Sep 20, 11:59 PM |
Open Colab → File → Save a copy in Drive. When finished, File → Download .ipynb and upload that file on Canvas. You may resubmit until the deadline. You may work in a team of at most 4 people. Even in a group, each person finishes their own notebook and submits it on Canvas. Name your teammates in the Teammates comment cell near the top of the notebook.
|
Course Structure - 5 Progressive Learning Phases
Phase 1: Foundations (Weeks 1-4) - Raschka Ch.1-6
Build LLMs from scratch: tokenization, attention mechanisms, transformer architecture, pre-training, and supervised fine-tuning
Phase 2: Core Implementation (Week 5) - Raschka Ch.7
Master instruction fine-tuning techniques to align models with human instructions and preferences
Phase 3: Large-Scale Training Systems (Weeks 6-10)
Advanced training infrastructure: scaling laws, distributed training, efficient attention variants, and memory optimization for production systems
Phase 4: Prompting & Tool Integration (Weeks 11-12)
Systematic prompt design, chain-of-thought reasoning, RAG systems, and automatic prompt optimization techniques
Phase 5: Alignment & Inference Optimization (Weeks 13-14)
RLHF implementation, constitutional AI, human preference learning, efficient inference, and inference-time scaling
Currently Available: Lectures 1 to 6 are posted. Later notebooks will be posted as we get to them.
Required Textbook
Build a Large Language Model (From Scratch)
Sebastian Raschka
Manning Publications, 2024
The only required purchased book. Hands-on LLM implementation from scratch.
Useful Resources
Optional starting points for the first meeting (Overview). Tokenizers, later papers, and assignment files will be added when those sessions open.