Course Syllabus

STAT545 Applied Stochastic Processes

About the course

Stochastic processes are indexed collections of random variables used to describe phenomena in which a dependence structure arises from evolution across time (or space). Markov processes, in particular, are stochastic processes in which dependence is local: given the current state (or values on a separating boundary), the future (outside the boundary) is conditionally independent of the past (or interior history). Markov processes have rich applications in epidemiology, finance, biology, social science, engineering, chemistry, and beyond, and they are also important in statistics. In particular, Markov chain Monte Carlo (MCMC) methods are central to modern Bayesian statistics as a means to approximate complex posterior distributions for Bayesian inference via simulation. This course is a graduate-level introduction to Markov processes and Markov chains, covering four key areas: discrete-time models, continuous-time models, MCMC, and, briefly, Brownian motion and Gaussian processes. Students can expect to learn core concepts and probabilistic language for describing Markov processes, gain exposure to common models and estimation methods, and explore applications.

Instructor: Trevor Ruiz (he/him) [email]

Class meetings: 10:10am–12:00pm MW in 10-124

Office hours: MW 1:00pm–2:30pm in 25-236 and [by appointment] in 25-236 or via Zoom.

Drop ins are welcome but appointments are recommended/appreciated.

Catalog description: Properties, simulation, and application of stochastic processes. Discrete-time and continuous-time Markov chains, hidden Markov models, Poisson processes, Gaussian processes, continuous-state Markov processes. Markov chain Monte Carlo (MCMC) simulation methods and applications. Estimation methods for stochastic processes. Prerequisites: linear algebra (MATH206 or MATH 244); mathematical statistics (STAT426).

Textbook: Robert P. Dobrow, Introduction to Stochastic Processes with R. Wiley, 2016. An online version is available through the [Kennedy Library]. You can also purchase or rent a print or electronic copy through the bookstore or the [publisher’s website].

Learning outcomes:

• [L1] Apply Markov chains to model discrete-time phenomena and data.
• [L2] Apply Poisson processes, Gaussian processes, and continuous-time Markov chains and processes to model continuous-time phenomena and data.
• [L3] Choose an appropriate probability model to analyze a random phenomenon.
• [L4] Conduct simulation analyses of probability models by writing original computer code, and write reports that summarize results of simulation analyses.
• [L5] Implement Markov chain Monte Carlo (MCMC) methods in a modern computing environment.
• [L6] Apply and interpret Markov chain model fitting to data, including hidden Markov models.

Assessments

Your attainment of learning outcomes will be measured by homework assignments, a series of mini-projects, participation, one midterm exam, and a cumulative final exam. These are described below, with the relative contributions to final grades indicated parenthetically.

• Homeworks (20%). Homework assignments will comprise short sets of textbook problems. These tend to focus on mathematical and/or computational problem-solving applications of course content and serve to reinforce key ideas through practice. One problem selected at random will be scored from each assignment.

• Mini-projects (30%). You will complete two mini-projects in groups of 2–3. Each project will explore an application, illustration, implementation, or extension of a topic discussed in class. All students will complete one project on discrete-time Markov chains; for the second project you may choose either MCMC or continuous-time Markov processes. Please see the tentative schedule for reference. For each project, your group will give a 15-minute in-class presentation/discussion on the due date and submit a 2-page written summary (or another deliverable by prior agreement) written for an audience of peers. Projects may be more applied (e.g., a data application, a worked example from the text or elsewhere, or a software tutorial/vignette) or more methodological (e.g., simulation experiments, an extension not covered in lecture, or an exposition of a particular model, estimation technique, or algorithm). I will provide general suggestions for the class and work with each group to determine an exact topic.

• Participation (10%). For each regular class meeting, I will solicit 1–2 volunteers to contribute a brief example related to the day’s material. Examples will by default be selected from the textbook, but external examples are welcome with advance coordination and approval. Discussion of examples should take about 5–10 minutes but should not exceed 10 minutes. Students are expected to volunteer at least once during the quarter and are encouraged to volunteer twice. Meeting the “volunteer at least once” expectation will earn at least 7% toward your final grade; the remaining 3% will be based on overall contribution quality. Volunteering a second time does not automatically increase your score, but it provides an additional opportunity for strong contributions to be reflected in the quality component. Attendance may be considered when making end-of-quarter adjustments to your participation grade; adjustments will only be made in cases of more than one unexcused absence.

• Midterm exam (20%). One midterm exam will be given in Week 5 with two components: a written in class component administered on Wednesday, February 4, and a take-home component due Friday, February 6. The in-class component midterm will focus on conceptual and mathematical aspects of discrete-time Markov chains and their applications. The take-home component will focus on applications involving computation, simulation, and/or data analysis. The multi-day window for the take-home component is intended to provide schedule flexibility rather than signal an onerous time commitment; for a well-prepared student, it should require at most about two hours to complete.

• Final exam (20%). A cumulative final exam will be given at the time scheduled by the registrar: 10:10am–1:00pm on Wednesday, March 18, in our usual classroom. The final exam will be entirely written (i.e., no computational or data-analytic components requiring the use of software) and will be open-book and open-note.

Your scores will be recorded in Canvas for your reference along with an estimate of your running course total on a 0-100 scale. Tentatively, letter grades will span the following ranges: A (90, 100]; B (80, 90]; C (65, 80]; D (50, 65]; F [0, 50]. Please note these are rough estimates and subject to change without notice (though I do make an effort to notify the class of any substantial changes and will report final thresholds at the end of the quarter). Please also note that failure to adhere to course policies may result in a lower letter grade than would otherwise be assigned.

Tentative schedule

Subject to change at instructor discretion.

Week	Subject	Topics	Reading	Assignments (due)
1 (1/5)	Discrete-time Markov chains	Introduction to stochastic processes; transition probabilities and distributions	1.2, 2.1-2.4	None
2 (1/12)	Discrete-time Markov chains	Limiting and stationary distributions, communication classes, recurrence and transience	3.1-3.3	HW1 (M)
3 (1/20)	Discrete-time Markov chains	Long-run behavior: limit theorems, periodicity, and ergodicity	3.4-3.6, 3.8, 3.10	HW2 (M)
4 (1/26)	Discrete-time Markov chains	Estimation methods; hidden Markov models	NA	HW3 (M) MP1 (W)
5 (2/2)	NA	NA	NA	MP1 (M) Midterm (W)
6 (2/9)	Markov chain Monte Carlo (MCMC)	Metropolis-Hastings algorithm, Gibbs sampling, and other methods; diagnostics and applications	5.1-5.3	HW4 (M)
7 (2/17)	Continuous-time Markov chains	Homogeneous Poisson processes	6.1-6.3	HW5 (M)
8 (2/23)	Continuous-time Markov chains	Extensions of Poisson processes: nonhomogeneous processes, spatial point processes, applications	6.6-6.7	HW6 (M)
9 (3/2)	Continuous-time Markov chains	Transition function, transition rates, generator, long-run behavior	7.1-7.4	MP2 (W)
10 (3/9)	Continuous-state processes	Brownian motion and Gaussian processes	8.1-8.4	None

Tips for success

I want you to succeed in this course. Below are some simple but effective habits:

• find a buddy or form a study group

• do the reading (skim before class, read closely after class)

• use office hours to discuss general questions about material and concepts, not just homework help (but that too)

• prepare summary notes and try a few extra problems from the book before exams

• take notes in class, but listen too—exact transcripts aren’t necessary (since I post notes) and writing every detail can distract from following the overall narrative

If you find yourself falling behind at any point during the quarter, or feel you are struggling with the course, please come and talk with me. The sooner you reach out, the more options I’ll have to help you.

Policies

Time commitment

STAT545 is a four-credit course, which corresponds to a minimum time commitment of 12 hours per week, including class meetings, reading, assignments, and study time. I try not to assign work in substantial excess of this minimum, but you should expect to invest between 12 and 14 hours per week on average, with occasional overage due to exams, mini-projects, or simply difficult material requiring more careful reading. While I aim to distribute workload as evenly as possible throughout the quarter and offset expected overages by adjusting assignment schedules, you should allow an extra hour or two in your schedule to accommodate variability if possible. Since class meetings account for only four hours per week, I recommend that you budget approximately 10 hours per week outside of class meetings to study and complete assignments. (This recommendation includes a buffer for week-to-week variation.) Please let me know if you are regularly exceeding these amounts or if you need help managing your time efficiently in the course.

Attendance and absences

Regular attendance is essential for success in the course and required per University policy. Absences should be excusable, but you do not need to notify me unless you anticipate an extended absence or will miss an in-class assessment; I trust you to adhere to Cal Poly norms and policies regarding class attendance. Please note, however, that frequent unexplained absences may negatively impact your course grade.

Classroom environment

I support Cal Poly’s commitment to building an inclusive learning environment where all students can succeed. To that end, I strive to create a classroom in which every student is treated with respect and dignity, regardless of background, beliefs, opinions, identity, or the many visible and nonvisible differences within our community. I want you to feel comfortable in class, especially when sharing your perspective, asking questions, and engaging in discussion with me and with your peers. All members of this class are therefore expected to contribute to a respectful, supportive, and inclusive climate. I expect you to treat others with respect, even (and especially) when you disagree with or do not understand their perspective. I hold myself to the same standard, and I expect the same of every other student in the class. If you experience any form of disrespect or discrimination, small or large, please speak with me.

Collaboration

Collaboration with classmates is encouraged. If you work with a group on homework problems, you are expected to be an active contributor and prepare your own solutions in your own words and writing, and by submitting your work you are attesting that you have met this expectation. You should not distribute or accept copies of written solutions under any circumstances.

Use of AI

I encourage the use of AI to support, but not replace, critical thinking. If I do not want you to use AI for a particular task or assignment, I will say so explicitly. Otherwise, you may use AI at your discretion for supportive tasks, such as clarifying concepts, drafting exploratory code, and generating practice problems. For example:

• Concept review: “Remind me how irreducible is defined and what it means intuitively.”

• Problem-solving strategies: “Give me a few ways to check whether a Markov chain is irreducible, with examples.”

• Coding help: “Provide simple R code to simulate a bounded 2D random walk.”

• Extra practice: “Create a few practice problems like this one, but vary the numbers and context.”

As a rule of thumb, AI use for secondary tasks is usually acceptable, but AI use for the direct resolution of primary tasks (e.g., producing full solutions to assigned problems or writing work you submit as your own) is not acceptable and may constitute plagiarism. To avoid crossing this line, don’t paste full homework/exam prompts into AI tools, regardless of your other instructions—many will simply solve them whether you ask for a solution or not, and this is hard to “unsee.” If you’re stuck, make up a structually similar problem or ask for general solution strategies (like the second example prompt above). Remember that you’ll need to demonstrate understanding without AI on exams. For further guidance on responsible use, see the CSU AI Commons page on [ethical AI use for students].

Lastly, I recommend using Cal Poly’s ChatGPT Edu for privacy reasons; under Cal Poly’s licensing agreement, conversation history and user data are protected and are not used to train future models.

Communication and email

I encourage you to ask questions in class and during office hours, since that is the only certain means of obtaining a response within a guaranteed time frame.

I respond to most email within 24 weekday hours, but I cannot guarantee this response time and I occasionally miss messages altogether (though I try not to). I rarely answer emails at night or on weekends, so while you are welcome to write me outside of business hours, please don’t expect a reply until the following business day. I also sometimes get behind on answering emails, so please wait a few days (preferably one week if it’s not pressing) before sending a follow-up or reminder.

Please do not ask technical questions about assignments by email.

Late and missing work

I understand that unexpected circumstances may arise and require you to temporarily rearrange your priorities and commitments on occasion during the quarter. You may, at any time during the quarter and without notice or penalty, use the following personal exceptions:

• turn in one homework assignments up to one week late

• miss one homework assignment altogether

Once your personal exceptions are exhausted, homework assignments turned in up to one week late will be awarded 50% credit unless an extension is granted in advance, and missing homework assignments will be treated as zeros. If you miss no homework assignments, your lowest score will be dropped.

No other late work will be accepted unless an exception to this policy is granted. I will consider exceptions for personal and medical emergencies or other similarly unforeseeable circumstances.

Requests for reassessment of previously graded work

I make my best effort to assess your work accurately and apply assessment criteria consistently across the class. While I sometimes do so imperfectly, I am also aware that granting adjustments to scores or grades can disadvantage more reticent students and favor those more comfortable approaching me about credit awarded on course assessments. So, in consideration of maintaining fairness, I ask that you limit requests for reassessment to clear mistakes, discrepancies, or oversights. And in the interest of maintaining accuracy, I also ask that you please do let me know if you think such an error may have occurred; if so, I will then ascertain whether other students in the class are owed credit and adjust evenly across the whole class to the best of my ability.

Please raise any possible issues with assessment in a timely manner (i.e., within one week of recieving graded work) and not at the end of the quarter. While I may consider late requests to review previously graded work at my discretion, I make no guarantee that I will do so.

Changes to scores or final grades

Per University policy, faculty have final responsibility for grading criteria and grading judgment and have the right to alter student assessment or other parts of the syllabus during the term. It is not appropriate to attempt to negotiate scores or final grades for any reason. Once the term has concluded, final grades will only be changed in the case of clerical errors, without exception. If you feel your grade is unfairly assigned at the end of the course, you have the right to appeal it according to the procedure outlined here.

Accommodations

It is University policy to provide, on a flexible and individualized basis, reasonable accommodations to students who have disabilities that may affect their ability to participate in course activities or to meet course requirements. Accommodation requests should be made through the Disability Resource Center (DRC).

Conduct and Academic Integrity

You are expected to be aware of and adhere to University policy regarding academic integrity and conduct. Detailed information on these policies, and potential repercussions of policy violations, can be found via the Office of Student Rights & Responsibilities (OSRR).