Description

This course introduces decision making under uncertainty from a computational perspective and provides an overview of the necessary tools for building autonomous and decision-support systems. Following an introduction to probabilistic models and decision theory, the course will cover computational methods for solving decision problems with stochastic dynamics, model uncertainty, and imperfect state information. Topics include Bayesian networks, influence diagrams, dynamic programming, reinforcement learning, and partially observable Markov decision processes. Applications cover air traffic control, aviation surveillance systems, autonomous vehicles, and robotic planetary exploration.

Prerequisites: basic probability (e.g., CS109) and fluency in a high-level programming language (e.g., CS106A). The appendices in the course textbook provides some review material as well as an introduction to the essential elements of the Julia programming language.

In-person attendance is not required for the course (including lectures, problem sessions, and exams). There is a mixture of in-person and online office hours.

Lectures

Lectures will be on Tuesdays and Thursdays, 9am to 10:20am in NVIDIA auditorium starting September 26, 2023. Students are welcome to attend in person live, participate through the live stream, or watch recordings offline. Registered students taking another course that is offered at the same time is not an issue. Videos (both live stream and recorded) are available through Panopto, which is linked to within Canvas.

The course is also available to the public through the Stanford Center for Professional Development.

The textbook serves as the official lecture notes.

Video cameras located in the back of the room will capture the instructor presentations in this course. For your convenience, you can access these recordings by logging into the course Canvas site. These recordings might be reused in other Stanford courses, viewed by other Stanford students, faculty, or staff, or used for other education and research purposes. Note that while the cameras are positioned with the intention of recording only the instructor, occasionally a part of your image or voice might be incidentally captured. If you have questions, please contact a member of the teaching team.

Major Themes

• There is a principled mathematical framework for defining rational behavior.
• The computational techniques discussed in class can lead to superior decisions that are sometimes counterintuitive.
• Successful application of these principles depends on the choice of representation and approximation.
• The same computational approaches can be applied to very different application domains.

Learning Outcomes

• You will gain a broad fundamental understanding of the mathematical models and solution methods for decision making (exercises, three quizzes).
• You will be able to implement and extend key algorithms for learning and decision making (two programming projects).
• You will be able to identify an application of the theory in this course and formulate it mathematically (proposal).
• You will gain a deep understanding of an area of particular interest and apply it to a problem (final project).
• You will be able to critique approaches to solving decision problems (peer review).

Units

AA228 will be offered for 3 or 4 units for either a letter or credit/no credit grade. Students registering for the 4 unit version of the course will be required to spend at least 30 additional hours extending their course project and preparing the paper for a peer-reviewed conference submission (actual submission is not required). The grade breakdown listed in the “Grading” section is the same regardless of whether the class is taken for 3 or 4 units.

Please confirm that you are registered for the desired number of units and the desired grading basis. There are deadlines for making these changes set by the registrar.

Textbook

This year we will be using the textbook titled Algorithms for Decision Making, MIT Press, 2022. Printed copies are available for purchase or you may download it for free as a PDF. Details are here.

Problem Sessions

Problem sessions will be held in person on Wednesdays from 4:00pm-5:00pm in Bishop Auditorium (located on the first floor of Lathrop library). You are highly encouraged to go to the in-person problem session, but in case you can’t make it, we also live-stream and record the problem session. The links to the live stream can be found under the “Zoom” tab in Canvas. Recordings of problem sessions can be found under the “Panopto Course Videos” tab in Canvas. Some problem sessions will be used to review for quizzes. Please bring your questions! Problem session slides and notes can be found here.

Discussion

Class discussions are held on Ed. You should be automatically registered through Canvas.

High-Resolution Feedback

To ensure that we are providing the best learning experience for our students, we will be sending out a survey on a weekly basis to a randomly selected group of students. Please note that each student will only receive one request to participate in the survey during the quarter. We greatly value the input and feedback of all our students and would highly appreciate your participation when it is your turn.

Grading

Projects: 20%

• 0% Project 0
• 10% Project 1
• 10% Project 2

Midterms: 45%

• 0% Quiz 0
• 15% Quiz 1
• 15% Quiz 2
• 15% Quiz 3

Final project: 35%

• 5% Proposal
• 5% Status Update
• 20% Paper or Video
• 5% Peer Review

Grades are posted on Canvas.

This course offered for  for either a letter or credit/no credit grade. If taking for credit/no credit, credit will be given to students who score a C- or higher (at least 70% in the course). We will use the standard breakdowns in the table below. We will round fractional percentages in your favor. Every year, a few students are awarded an A+ after careful consideration for demonstrating mastery beyond what is expected in this class; it is not determined solely based on percentage.

Late Policy

All assignments are due at 5pm on Fridays.

Because of unexpected events, illnesses, work commitments, etc., there is a 0% penalty for 72 hours (no questions asked) after each assignment deadline (not quizzes!), after which you receive 0 credit. Note that late submissions will not qualify for the leaderboard for programming projects 1 and 2.

Quizzes submitted past their deadlines will receive 0 credit.

Quiz Reflections

The purpose of this policy is not solely to improve your scores but to enhance your understanding of the course material. Learning from your mistakes is an essential part of the educational journey. To encourage this process and help deepen your understanding of the course material, we are planning a trial run of a policy that allows students to reflect on their quizzes and earn up to half of the missed points back. This policy was inspired and adapted from a similar policy in CS161.

Opportunity: If you score below 70% on a quiz, you have the opportunity to write a reflection on the problems you missed. The adjusted score will be your original score plus up to half of the total points you missed, with a maximum adjusted score of 70%.

Reflection Guidelines: Your reflection should be in PDF format and include the following:

• Your original answer to the question
• The number of points you originally earned for that question (e.g. 3 / 5)
• The rubric item(s) for that question as specified on Gradescope (e.g. “Right answer but significant error in explanation”)
• If applicable, any additional point adjustments or rubric items.
• A detailed explanation of your mistake, including any incorrect assumptions or incomplete solutions.

When writing your reflection, dedicate a significant portion to explaining, in detail, where you went wrong in your original response. Identify the specific mistake or errors in your original answer. Go beyond surface-level identification and delve into the root cause of your mistake. Ask yourself why you made this error. Did you misinterpret the question? Did you overlook a critical piece of information? If your mistake was due to assumptions or misconceptions, explicitly state what these assumptions or misconceptions were. If your original response was incomplete, explain what critical steps or components you omitted. Walk through the problem step-by-step and clarify where you should have elaborated further.

Deadline: You must submit your reflection by the specified deadline, which we will announce on Ed. Details on the submission process will be provided shortly after the first quiz.

Notification: The adjusted points will be updated in Gradescope.

If you have any questions or need further clarification regarding this policy, please don’t hesitate to reach out to the teaching team.

Students with Disabilities

Students who may need an academic accommodation based on the impact of a disability must initiate the request with the Office of Accessible Education (OAE). Professional staff will evaluate the request with required documentation, recommend reasonable accommodations, and prepare an Accommodation Letter for faculty dated in the current quarter in which the request is made. Students should contact the OAE as soon as possible since timely notice is needed to coordinate accommodations. The OAE is located at 563 Salvatierra Walk (phone: 650-723-1066, URL: https://oae.stanford.edu/). Please submit OAE letters by Friday, 29 September to asmar@stanford.edu.