Course: Math 407, Probability Theory I, Fall 2020, Section 39632
Prerequisite: MATH 226 or MATH 227 or MATH 229
Course Content: Probability spaces, discrete and continuous distributions, moments, characteristic functions, sequences of random variables, laws of large numbers, central limit theorem, special probability laws.
Syllabus: here. Last update: 12 August 2020

Instructor: Steven Heilman, stevenmheilman(@-symbol)gmail.com
Office Hours: Mondays, 1030AM-1130AM and 430PM-530PM, on zoom [link posted on blackboard]
Lecture Meeting Time/Location: Mondays, Wednesdays, and Fridays, 12PM-1250PM, on zoom [link posted on blackboard]
TA: Andrew Lowy, lowya(@-symbol)usc.edu
TA Office Hours: Mondays 7PM-9PM Thursdays 8PM-9PM, held online in the Math Center.
Discussion Section Meeting Time/Location:

• 39520, Tuesdays and Thursdays, 2PM-250PM, on zoom [link posted on blackboard]
• 39521, Tuesdays and Thursdays, 3PM-350PM, on zoom [link posted on blackboard]

TA Office Hours: Occur in the Math Center.
You are not required to buy a textbook. Free lecture notes are provided below.
Recommended Textbook: D. P. Bertsekas and John N. Tsitsiklis, Introduction to Probability, 2nd edition. (The book is freely available online)
Another Recommended Textbook: Sheldon Ross, A First Course in Probability, any edition. (The book is freely available online)
Another Recommended Textbook: Elementary Probability for Applications, Durrett.

First Midterm:  Friday, September 18, 12PM-1250PM  [For students in Asia/Pacific time zones, the exam is Friday, September 18, 12AM-1250AM PST.]
Project Proposal due: Thursday, October 8, 2PM
Second Midterm: Friday, October 23, 12PM-1250PM  [For students in Asia/Pacific time zones, the exam is Friday, October 23, 12AM-1250AM PST.]
Progress Report due: Thursday, October 29, 2PM
Final Report due: Friday, November 13, 5PM
Final Exam: Friday, November 20, 11AM-1PM.  [For students in Asia/Pacific time zones, the exam is Thursday, November 19, 11PM PST to Friday, November 20, 1AM PST.]
Other Resources: Supplemental Problems from the textbook. An introduction to mathematical arguments, Michael Hutchings, An Introduction to Proofs, How to Write Mathematical Arguments

Zoom Classroom Conduct: Students should attend zoom lectures in a considerate way and abide by the following rules of decorum. Failure to do so could result in a diminished participation grade.  It is preferable (though not required, for equity reasons) that all students have a webcam on during the lecture.
Zoom Security: The zoom links posted on blackboard should not be shared with anyone. You must log into zoom with your USC email address. No one will be admitted to the lecture from the "waiting room" (if you are in the waiting room, you did not log in with your USC email address).
Zoom Technical Support: Technical support for undergraduate students is provided through USC's ITS. Below is the contact information.
Undergraduate Student Technology Support
Portal: https://itsusc.service-now.com/its_sp
Phone: 213-740-5555
Email: consult@usc.edu

Lecture Recording: Zoom lectures will be recorded and posted on the blackboard site. It is USC policy to prohibit the sharing of any recording of course lectures with others. Similarly, you should not create your own recording of the lectures.
Time Zone Issues: If the course lectures, office hours, or exam schedules occur outside the range of 7AM-10PM in your current time zone, please alert me to this fact as soon as possible. Late notification of such an issue (e.g. the day before an exam) may result in a denied rescheduling request.  If the course lectures occur outside the range of 7AM-10PM in your current time zone, then you will automatically get a 100% participation grade.

Email Policy:

• My email address for this course is stevenmheilman@gmail.com
• It is your responsibility to make sure you are receiving emails from stevenmheilman@gmail.com , and they are not being sent to your spam folder.
• Do NOT email me with questions that can be answered from the syllabus.

Exam Procedures: The midterms will be one-hour timed exams, to be submitted on blackboard (a 50 minute exam, with 10 minutes designated for uploading the exam).  In the midterm exams, you are allowed to consult your homeworks, your notes, and your textbook, but these are the only resources you are allowed to use during the exams. So, you are not allowed to use the internet, internet searches, a friend or assistant, etc. Phones must be turned off. If you anticipate issues with a stable internet connection (for obtaining the exam), issues with obtaining a suitable exam environment, etc., please let me know as soon as possible and we can try to come up with a solution to these issues. Cheating on an exam results in a score of zero on that exam. Exams can be regraded at most 15 days after the date of the exam. This policy extends to homeworks as well. All students are expected to be familiar with the USC Student Conduct Code. (See also here.)
Disability Services: If you are registered with disability services, I would be happy to discuss this at the beginning of the course. Any student requesting accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure the letter is delivered to me as early in the semester as possible. DSP is located in 301 STU and is open 8:30am-5:00pm, Monday through Friday.
https://dsp.usc.edu/
213-740-0776 (phone)
213-740-6948 (TDD only)
213-740-8216 (fax)
ability@usc.edu

Discrimination, sexual assault, and harassment are not tolerated by the university. You are encouraged to report any incidents to the Office of Equity and Diversity http://equity.usc.edu/ or to the Department of Public Safety http://capsnet.usc.edu/department/department-public-safety/online-forms/contact-us. This is important for the safety whole USC community. Another member of the university community - such as a friend, classmate, advisor, or faculty member - can help initiate the report, or can initiate the report on behalf of another person. The Center for Women and Men http://www.usc.edu/student-affairs/cwm/ provides 24/7 confidential support, and the sexual assault resource center webpage sarc@usc.edu describes reporting options and other resources.

Exam Resources: Here are the exams I used when I previously taught a similar course: Exam 1 Exam 1 Solutions Exam 2 Exam 2 Solutions. Final Final Solutions. Exam 1 Exam 1 Solutions Exam 2 Exam 2 Solutions. Final Final Solutions. Exam 1 Exam 1 Solutions Exam 2 Exam 2 Solutions. Final Final Solutions. Here is a page containing old exams for another similar class. Here is a practice midterm (with solutions). Here are some more exams from a second quarter probability course: Exam 1 Exam 1 Solution Exam 2 Exam 2 Solution Final Final Solution. Exam 1 Exam 1 Solutions Exam 2 Exam 2 Solutions Final Final Solutions. Occasionally these exams will cover slightly different material than this class, or the material will be in a slightly different order, but generally, the concepts should be close if not identical.

Homework Policy:

• Homeworks are due at 2PM Thursdays, i.e. at the beginning of the first discussion session on Thursdays.
• Homeworks are submitted in blackboard, under the "Assignments" tab. You are allowed unlimited submission "attempts" for an assignment, but only the last submission will be graded. To avoid internet issues, I recommend making your first submission of an assignment well in advance of the deadline. (Note that phone tethering can also give you an internet connection to a computer.)
• Homeworks should be submitted as single PDF documents. One way to create a PDF document from paper homework assignments is the freely available Adode Scan App
• Late homework is not accepted.
• If you still want to turn in your homework late, then the number of minutes late divided by ten will be deducted from the homework score.  The exact deduction and rounding procedure is not guaranteed to be accurate.
• The lowest two homework scores will be dropped. This policy is meant to account for illnesses, emergencies, dropped internet connections, etc.
• You may not use the internet to try to find answers to homework problems.
• Do not submit homework via email.
• Collaboration on homeworks is allowed and encouraged.
• All homework assignments must be written by you, i.e. you cannot copy someone else`s solution verbatim.

Grading Policy:

• The final course grade is weighted as the larger of the following two schemes.
  • Scheme 1: class participation (3%), homework (17%), project proposal (3%), project progress report (7%), final project report (20%), the first midterm (15%), the second midterm (15%), and the final (20%).
  • Scheme 2: class participation (3%), homework (17%), project proposal (3%), project progress report (7%), final project report (25%), largest midterm grade (20%), and the final (25%).
  The grade for the semester will be curved. However, I do not "curve down" since anyone who exceeds my expectations in the class by showing A-level performance on the exams and homeworks will receive an A for the class.
• If you cannot attend one of the exams, you must notify me within the first two weeks of the start of the quarter. Later requests for rescheduling will most likely be denied.
• Class participation is not the same as attendance.  I will never explicitly take attendance, but I will notice if someone is frequently absent.  Things that increase your class participation grade include: asking good questions, paying attention in class, showing up on time or early to class, etc. Things that decrease your class participation grade include: excessive talking or disruptions during class, frequent absences, excessive texting/smartphone usage in class, frequent tardiness, etc.
• You must take the final exam to pass the course.

Final Project Guidelines:
The final project is an opportunity to explore your interests and learn something e.g. that we didn't have time to cover in class. A project could begin with an interesting question or a well-known problem, and perhaps lead to a probabilistic model of some phenomena that interests you, investigating or implementing various algorithms, conducting an empirical analysis, etc.

The goal is to say something interesting about a problem in probability, broadly construed to perhaps include statistics, machine learning, etc. You could read some section of a book that we will not cover in the class, and report the most salient facts there, reproducing/streamlining the proofs there. You could perhaps develop new methodology for an existing problem or application that has no fully satisfactory solution. You could alternatively tackle a new problem or application with existing methodology; in this case, you should identify one or more questions without satisfactory answers in your chosen domain and explore how the methodology can help you answer those questions. You could consider some statistical problem and draw inspiration from particular datasets, but your focus should rest not on the data itself but rather on the questions about the world that you can answer with that data.

You may work alone or in a group of two; the standards for a group project will be twice as high. In certain cases I might approve a group of three, but this is unlikely.

We strongly encourage you to come to office hours to discuss your project ideas, progress, and difficulties.

Final Project Milestones: In all cases, ideally you would use LaTeX, but you are not required to use LaTeX.

I: Project Proposal. By this first milestone, you should have selected a question or problem of interest, found some notes or textbooks that discuss your project subject matter, identified relevant data sources (if applicable), begun exploring the literature surrounding the question/topic, and discussed your ideas with the course staff. Your project proposal deliverable is a 1/2 - 1 page report describing the question or problem you intend to tackle, why this question is important or interesting, prior work on this problem, what data you intend to use in your analyses (if applicable), and the principal challenges that you anticipate (if applicable).

If you would like to receive feedback about particular aspects of your proposal, please indicate this in your submission.

I can try to help in problem selection. Ideally, the problem should be something you are very interested in. As such, it might be helpful to first tell me about your interests (maybe after class or in office hours), and we can try to think of something to work on.

II: Progress Report. By this second milestone, you should have some initial results to share; for example, you may have read the section of a book that interests you, you may have implemented and evaluated the performance of some algorithm on a dataset, you may have constructed a probabilistic model for your problem at hand, or you may have conducted an initial study with simulated data to better understand the properties of certain methods, etc.

Your progress report deliverable is a write-up of no more than 2 pages (single-spaced; not including references) describing what you have accomplished so far and, briefly, what you intend to do in the remainder of the term. You should be able to reuse at least part of the text of this milestone in your final report.

III: Final Report. Your final project report (not including acknowledgements and references) should be around 5-8 pages in length (using at most 12 point font, maximum 1 inch margins, and single-spaced) and should follow a typical scientific style (with abstract, introduction, etc.). The write-up should clearly define your problem or question of interest, review relevant past work, and introduce and detail your approach. A comprehensive empirical evaluation could follow, or some proofs of some results, along with an interpretation of your results. Any elucidation of the theoretical properties of an empirical method under consideration is also welcome.

If this work was done in collaboration with someone outside of the class (e.g., a professor), please describe their contributions in an acknowledgements section.

Some Project Ideas:

Machine Learning/ Deep Learning

• Investigate some Reinforcement Learning Concepts with OpenAI
• Read about/ code up/ prove a few things about the Perceptron algorithm, one of the most basic classification algorithms in machine learning (see e.g. Section 2.1 of my notes here)
• Read and report about the following paper: Mastering the game of Go without human knowledge. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel & Demis Hassabis, here
• Use a statistical model to try to predict winners of sports tournaments such as NCAA March Madness
• Learn about facial recognition software (such as eigenfaces); try to code up your own version of this software.
• Try to implement your own deep learning algorithm to solve some problem, such as classical handwriting recognition, etc.

Proofs of the Central Limit Theorem

• Re-prove some general versions of the central limit theorem (such as the Lindeberg-Feller CLT), and/or prove the original CLT with several proof methods (see e.g. Section 3.4 of my notes here)

Stochastic Processes

• Investigate some properties and/or re-prove some things and/or do some simulations of one or two basic stochastic processes, such as random walks, Brownian motion, Poisson Process, Markov chains, etc. (see e.g. these notes)

Polling/ Elections

• Look into the methodology for the 538 presidential forecast for the 2020 election.
• Create a probabilistic model of voter behavior in the 2020 election, and try to predict the outcome.

Concentration of Measure

• Read about and re-prove some facts such as the Hoeffding inequality, and related inequalities, with applications to some other problems.

Probabilistic Algorithms

• Investigate some particular probabilistic algorithm of interest, such as how to use random sampling to integrate a function (which is used e.g. in computer graphics)

Tentative Schedule: (This schedule may change slightly during the course.)

Advice on succeeding in a math class:

• Review the relevant course material before you come to lecture. Consider reviewing course material a week or two before the semester starts.
• When reading mathematics, use a pencil and paper to sketch the calculations that are performed by the author.
• Come to class with questions, so you can get more out of the lecture. Also, finish your homework at least two days before it is due, to alleviate deadline stress.
• Write a rough draft and a separate final draft for your homework. This procedure will help you catch mistakes. Also, it might be beneficial to typeset your homework. Learning LaTeX is a good skill to have for doing mathematics. Here is a template .tex file if you want to get started typesetting.
• If you are having difficulty with the material or a particular homework problem, review Polya's Problem Solving Strategies, and come to office hours.

Homework

• HW0 HW1 HW2 HW3 HW4 HW5 HW6 HW7 (solutions removed)

Homework .tex files

• HW0 HW1 HW2 HW3 HW4 HW5 HW6 HW7

Exams

• Exam 1 Exam 1 Solution Exam 1v2 Exam 1v2 Solution
• Exam 2 Exam 2 Solution Exam 2v2 Exam 2v2 Solution
• Final Final Solution Finalv2 Finalv2 Solution

Homework Digest

• dig1

Supplementary Notes

• 407 Lecture Notes