Realistic (& Advanced) Image Synthesis
Fall 2024
This course presents modern mathematical models of lighting and shading, and the numerical methods and algorithms needed to solve them and generate beautiful realistic images.
Technical content will be delivered using a combination of in-class lectures, tutorial/lab sessions, and (optional) supplemental reading. Learning objectives will be solidifi ed and evaluated through Python coding assignments (relying heavily on parallelization, user interfacing, and windowing with TaiChi.)
1 Administrative Details
The weekly schedule includes two 80-minute lectures on Wednesdays and Fridays, from 11:35 am to 12:55 pm in Trottier Building 1100 (TR 1100), and a tutorial/lab session. Tutorial/lab sessions (intertwined) depend on the section you registered in — please refer to Minerva to ensure you attend the right session.
|
ECSE Course Code
|
Credit Value
|
|
ECSE 446
|
3 Credits
|
|
ECSE 546
|
4 Credits
|
2 Course Overview
This course dives deep into the math and algorithms behind visual effects in modern feature ilms and games. After an overview of basic radiometry, we present the mathematical models of appearance and light's propagatation in physical environments. We then develop algorithms to help simulate these processes, generating realistic images.
You will code state-of-the-art numerical methods used in visual effects, video games and machine learning. During the tutorial/lab sessions, we will direct you through common implementation stumbling points; however, students are those primarily responsible for overcoming the modeling and coding challenges behind the learning objectives. Please refer to the topics and programming assignments we'll cover in the class, below. For many students, this course will provide a first exposure to Python and — arguably more importantly — vectorization using numpy and parallelization using taichi. While the teaching staff will provide support during the tutorial/lab sessions, students are expected to commit to (potentially signifi cant) independent at-home coding and debugging efforts during the semester.
|
Physics-based Shading
|
Light Transport Algorithms
|
|
What is light? What are the physical
units and intuitive interpretations behind those quantities most commonly used to measure and represent light?
We'll very quickly establish a common language to use when discussing the physics of light. The emphasis here (as with the entire course) is to build an
intuition, rather than unnecessarily complex mathematical constructs.
|
We'll derive a rendering equation that governs the steady-state equilibrium of light energy in a scene. We will generate realistic images of 3D worlds by numerically solving this equation.
Speciically, you'll code numerical solvers based on Monte Carlo integration, the leading family of strategies employed in the graphics industry. The methods we explore also apply to other
problems in engineering, physics and statistics: as such, you'll develop a broadly applicable "numerical toolbox".
|
3 Requirements
We outline some high-level requirements and expectations, below.
3.1 Prerequisites (only enforced for ECSE students)
ECSE 202: Introduction to Software Development
ECSE 205: Probability and Statistics for Engineers and
COMP 250: Introduction to Computer Science
Generally speaking, we expect students to be comfortable with basic probability concepts (discrete and continuous), differential calculus and linear algebra. You'll be coding quite a bit in this course, so comfort with the build-implement-debug workfl ow, data structures and algorithms, and basic software engineering will come in handy.
The assignments will be implemented in Python, relying heavily on the TaiChi visualization and parallel programming library.
3.2 Textbook (available online: HTML and PDF.)
Physically Based Rendering: From Theory to Implementation (4th ed.) by Pharr, Humphreys and Jakob
This text is the comprehensive reference on realistic image synthesis. The book is packaged with its own open-source renderer (PBRT), and this implementation is exposed in a literate programming style. throughout the book.
At over 1500 pages, we only expect students to refer to this book as a supplemental resource for topics covered in class. You can access an e-publication version online for free in a web-friendly format and through the McGill library network.
3.3 Equipment
You do not need to buy special computing equipment for this class (i.e., any computer that's less than six years old should suffi ce). All of our assignments will rely on a baseline Python and TaiChi environment.
3.4 Time
Attending and participating in lectures is strongly correlated with the successful retention of the learning objectives: we encourage attending every lecture. It is not unreasonable to plan to invest nine (9) hours per week for the 3-credit course or 12 hours per week for the 4-credit version (this includes lecture, tutorial/lab and at-home coding time) on average.
For assignments, it's reasonable to spend roughly 70% of your time coding — the remainder will be split between planning, debugging and testing. Start early and schedule appropriately. Attending tutorials/labs with your prepared questions is a good way to skirt any avoidable diffi culties.
4 Evaluation
Students will be evaluated based on four (4) programming assignments, to be completed individually. ECSE 546 students will have additional deliverables assigned to them in the assignments.
Your score relies on the successful completion of the programming tasks, each described below. There is no midterm or final exam.
A detailed breakdown of each of these graded components follows.
4.5 Coding Assignments
Assignments will use the Python programming language.
Assignments are to be completed individually.