Hidden Markov Models 03: Reasoning with a Markov Model

Summary

This lecture introduces observable Markov models using a weather prediction example, where states represent rainy, cloudy, and sunny conditions. The lecturer explains the Markov property and the transition matrix, which defines the probabilities of moving between states. By applying these concepts, the lecturer demonstrates how to calculate the probability of a sequence of weather conditions over multiple days. The approach uses the Markov property to break down complex probabilities into manageable calculations, ultimately providing insight into predicting future weather with limited information.

Takeaways

• 😀 The lecture introduces the concept of an Observable Markov Model as the foundation for Hidden Markov Models.
• 😀 A Markov Model is based on the Markov Property, where the prediction of future states only depends on the current state, not past states.
• 😀 The weather model in the lecture consists of three states: rainy or snowy (s1), cloudy (s2), and sunny (s3).
• 😀 At noon each day, a weather observation is made, which determines the state of the system for that day.
• 😀 The transition matrix (A) defines the probability of moving from one state to another on the next day, with each row summing to 1.
• 😀 The probability of staying in the same weather state from one day to the next is higher than transitioning to a different state (e.g., 80% chance of staying sunny).
• 😀 The goal is to calculate the probability of a specific sequence of weather states over multiple days, using the transition matrix.
• 😀 The probability of a sequence of observations is calculated by multiplying the probabilities of transitions between states for each day.
• 😀 The process uses the Markov Property, where the sequence of probabilities for state transitions is dependent only on the current state, not the entire history.
• 😀 The final probability of the weather sequence given the model is very low (1.536 × 10^(-4)), but this is due to the large number of possible state sequences over seven days.

Q & A

What is an observable Markov model?

An observable Markov model is a type of Markov model where the system's states can be observed directly. It serves as the foundation for building a hidden Markov model, where the states are not directly observable.

What are the three weather states used in this example?

The three weather states in the example are: s1 (rain or snow), s2 (cloudy), and s3 (sunny).

What is the Markov property in the context of this weather prediction model?

The Markov property states that the probability of transitioning to the next state (weather) depends only on the current state, and not on the previous states. This means the prediction for the weather on any given day can be made based solely on the weather of the previous day.

How is the transition matrix used in this example?

The transition matrix represents the probabilities of moving from one state to another. Each row of the matrix corresponds to the current state, and the entries in the row represent the probability of transitioning to the possible next states.

What does the transition matrix look like in this example?

In the example, the transition matrix contains the following probabilities: from sunny (s3) to sunny (0.80), from cloudy (s2) to sunny (0.20), from rainy (s1) to rainy (0.40), from cloudy (s2) to cloudy (0.60), etc.

How can the probability of a specific weather sequence be calculated?

The probability of a specific weather sequence is calculated by multiplying the transition probabilities of each state transition in the sequence, starting from the initial state. In the example, we calculate the probability of a seven-day sequence using the transition matrix.

What is the significance of the initial state being sunny (s3) in the example?

The initial state being sunny (s3) is significant because it sets the starting point for the probability calculations. It is assumed to have a 100% probability of being sunny on the first day (T=1).

What does the final probability of 1.536 × 10⁻⁴ represent?

The final probability of 1.536 × 10⁻⁴ represents the likelihood of observing the specific sequence of weather conditions (sun, sun, rain, etc.) over the next seven days, starting from a sunny state (s3), based on the transition probabilities.

Why is the probability of the sequence relatively low?

The probability is relatively low because there are many possible sequences that could occur over the next seven days. With three possible states per day, there are 3^7 different possible weather sequences, making the specific sequence in the example less likely.

What role does the Markov property play in simplifying the calculation of the weather sequence probability?

The Markov property simplifies the calculation by allowing the sequence of weather states to be broken down into a series of independent transitions, where the probability of each transition depends only on the current state. This reduces the complexity of calculating the overall probability of the sequence.