Stat 360 -- Introduction to Generative AI Winter 2025

Stat 360 course materials

Office Hours

Project title	Time	Location
Ryan Chen (TA)	Thursdays 3-4pm	Zoom: https://northwestern.zoom.us/j/3082623966
Professor Stadie	Posted on Canvas	Zoom: https://northwestern.zoom.us/j/3799597115

Course Lectures

Lecture notes can be found on the course Canvas website.

Lecture	Date	Material	Readings
Week 1, Tuesday	January 7	Introduction	Perplexity, Perplexity 2, Linear Models
Week 1, Thursday	January 9	Attention	Attention is All You Need, Attention Mechanisms, Attention with code, The Illustrated Transformer
Week 2. Tuesday	January 14	Transformers	Intro to Transformers, Discussion, Blog post, Skip connections, Layer normalization, Byte-Pair Encoding
Week 2. Thursday	January 16	Coding Transformers
Week 3, Tuesday	January 21	BERT, GPT, LLAMA	Annotated GPT 2, GPT-2 paper, Adversarial attacks on GPT-2 LLAMA Paper, BERT, Understanding BERT
Week 3, Thursday	January 23	Prompt Tuning, chain of thought, hindsight chain of thought, backwards chain of thought, Graph of Thought, Tree of Thought, Training Chain-of-Thought via Latent-Variable Inference, prompt engineering	Language Models are Few Shot Learners, Zero shot chain of thought, LLMs are human-level prompt engineers, Tree of Thought, Chain of verification, Promptbreeder, Prompt Engineering Guide, Open-AI Prompting Guide
Week 4, Tuesday	January 28	Fine tuning, tool use, parameter-efficient fine tuning, LORA, Instruction Tuning (SFT), Neftune, quantization, Hugging face, fine tuning LLAMA	LORA, PEFT, Quantization, Quantization Blog, Alpaca, ToolEMU, NEFTune, Tool Use code, Model Calibration Mistral Fine Tuning, LLAMA fine tuning
Week 4, Thursday	January 30	ChatGPT and RLHF, rejection sampling, DPO, Gopher Cite	Stack LLAMA, Instruct GPT, PPO, DPO, RLHF References, TRL, Gopher Cite, Chain of hindsight
Week 5, Tuesday	February 4	No class
Week 5, Thursday	February 6	RAG, when to use RAG vs SFT, lexacagraphical vs semantic search, sentence transformers, Retrieval transformers and long term memory in transformers, RAG Code.	RAG, RAG code, Sentence Transformers, Rag Evaluation, Advanced RAG
Week 6, Tuesday	February 11	Multi-Agent LLMs	Wonderful Team, ReWoo, Reasoning with Language Model, LLM+P, ReACT, AgentVerse
Week 6, Thursday	February 13	Prompt optimization, reflection, steering
Week 7, Tuesday	February 18	GPTo1, LLMS + Tabular ML
Week 7, Thursday	February 20	Vision Models, Asking questions about images. Conditional layer norm, FILM, CLIP, BLIP, LAVA	BLIP, CLIP, Llava, FiLM
Week 8 Tuesday	February 25	Stable Diffusion, tuning stable diffusion, Diffusion models, DDPM, classifier-free guidance	Stable Diffusion, Illustrated Stable Diffusion, Understanding Stable Diffusion Diffusion, DDPM, Diffusion as SDEs, Classifier Free Guidance, Diffusion code, More low level code
Week 8 Thursday	February 27	Frontiers, using LLMs to help diffusion models by planning out images. SD Edit, Diffusion in robotics.
Week 9, Tuesday	March 4	Life is Worse with LLMs
Week 9, Thursday	March 6	No class. Complete take-home final exam

Homeworks and Due Dates

Project title	Date released	Due date
Assignment 1: Transformers	Jan 9	Jan 21
Assignment 2: Prompt Tuning	Jan 22	Jan 30
Assignment 3: SFT	Jan 31	Feb 11
Assignment 4: RLHF	Feb 13	Feb 24
Assignment 5: RAG	Feb 24	March 4
Take-home final exam	March 5	Due Friday, March 7th