About the Course
CSC 352/552 Generative AI is an advanced course that takes students from foundational LLM concepts to cutting-edge large-scale AI system implementation. Starting with Raschka's hands-on approach to building language models from scratch, students master tokenization, attention mechanisms, and transformer architectures. Later meetings cover prompting, retrieval-augmented generation (RAG), tool use, and related methods. Students complete hands-on projects implementing core LLM components.
Course Schedule
Course Materials
Available Lectures
| Lecture | Topic | Materials |
|---|---|---|
| -- | Course Introduction and Overview | Syllabus review, course expectations |
| 1 | Machine Learning Foundations | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 2 | PyTorch tensors and autograd | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 3 | PyTorch networks and training | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 4 | Tokenization | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 5 | Embeddings and data sampling | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 6 | Attention and Transformer Basics | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 7 | Causal and multi-head attention | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| -- | Review section 1: Tokenization to Attention | Open in Colab / Copy to Drive Download .ipynb Download PDF |
| 8 | Building GPT Architecture | Posted when we reach this meeting |
| 9 | GPT implementation | Posted when we reach this meeting |
| 10 | Model Training Pipeline | Posted when we reach this meeting |
| 11 | Training pipeline practice | Posted when we reach this meeting |
| -- | Review section 2: From the Transformer Block to a Talking GPT | Posted when we reach this meeting |
| 12 | Fine-tuning for Classification | Posted when we reach this meeting |
| 13 | Instruction Fine-tuning | Posted when we reach this meeting |
| 14 | Parameter-efficient fine-tuning / LoRA | Posted when we reach this meeting |
| 15 | Systematic prompting | Posted when we reach this meeting |
| Opt. | Chain-of-Thought and inference-time compute (optional reading; not covered in class) | Posted when we reach this meeting |
| 16 | RAG | Posted when we reach this meeting |
| Opt. | Vision-language models (optional reading; not covered in class) | Posted when we reach this meeting |
| 17 | Function calling and the agent loop | Posted when we reach this meeting |
| Opt. | MCP, skills, and hooks (optional reading; not covered in class) | Posted when we reach this meeting |
| Opt. | Loop engineering and graph engineering (optional reading; not covered in class) | Posted when we reach this meeting |
| 18 | Case study: Claude Code | Posted when we reach this meeting |
| Opt. | Case study: Grok Build (optional reading; not covered in class) | Posted when we reach this meeting |
| 19 | Final project kickoff | Posted when we reach this meeting |
| 20 | Final project work session | Posted when we reach this meeting |
| 21 | Project presentations / wrap-up | Posted when we reach this meeting |
Assignments
| Assignment | Topic | Due | Materials |
|---|---|---|---|
| 1 | Lectures 1 to 3 | Sun, Sep 20, 11:59 PM |
Open Colab → File → Save a copy in Drive. When finished, File → Download .ipynb and upload that file on Canvas. You may resubmit until the deadline. You may work in a team of at most 4 people. Even in a group, each person finishes their own notebook and submits it on Canvas. Name your teammates in the Teammates comment cell near the top of the notebook.
|
| 2 | Lectures 4 to 7 | Sun, Oct 4, 11:59 PM |
Open Colab → File → Save a copy in Drive. When finished, File → Download .ipynb and upload that file on Canvas. You may resubmit until the deadline. This is an individual assignment. Do the notebook yourself and submit your own .ipynb file.
|
| 3 | Lectures 8 to 11 | Sun, Nov 8, 11:59 PM |
Open Colab → File → Save a copy in Drive. When finished, File → Download .ipynb and upload that file on Canvas. A comment then gives you an interview link: explain a short piece of your own code from each problem on camera (5 minutes each, up to 5 tries); your score is posted after you finish it, by the deadline. You may resubmit until the deadline. You may work in a team of at most 4 people. Even in a group, each person finishes their own notebook and submits it on Canvas. Name your teammates in the Teammates comment cell near the top of the notebook.
|
Course Structure - 5 Progressive Learning Phases
Phase 1: Foundations (Weeks 1-4) - Raschka Ch.1-6
Build LLMs from scratch: tokenization, attention mechanisms, transformer architecture, pre-training, and supervised fine-tuning
Phase 2: Core Implementation (Week 5) - Raschka Ch.7
Master instruction fine-tuning techniques to align models with human instructions and preferences
Phase 3: Large-Scale Training Systems (Weeks 6-10)
Advanced training infrastructure: scaling laws, distributed training, efficient attention variants, and memory optimization for production systems
Phase 4: Prompting & Tool Integration (Weeks 11-12)
Systematic prompt design, chain-of-thought reasoning, RAG systems, and automatic prompt optimization techniques
Phase 5: Alignment & Inference Optimization (Weeks 13-14)
RLHF implementation, constitutional AI, human preference learning, efficient inference, and inference-time scaling
Currently Available: Lectures 1 to 6 are posted. Later notebooks will be posted as we get to them.
Required Textbook
Build a Large Language Model (From Scratch)
Sebastian Raschka
Manning Publications, 2024
The only required purchased book. Hands-on LLM implementation from scratch.
Useful Resources
Optional starting points for the first meeting (Overview). Tokenizers, later papers, and assignment files will be added when those sessions open.