Skip to main content
Home
plus.maths.org

Secondary menu

  • My list
  • About Plus
  • Sponsors
  • Subscribe
  • Contact Us
  • Log in
  • Main navigation

  • Home
  • Articles
  • Collections
  • Podcasts
  • Maths in a minute
  • Puzzles
  • Videos
  • Topics and tags
  • For

    • cat icon
      Curiosity
    • newspaper icon
      Media
    • graduation icon
      Education
    • briefcase icon
      Policy

      Popular topics and tags

      Shapes

      • Geometry
      • Vectors and matrices
      • Topology
      • Networks and graph theory
      • Fractals

      Numbers

      • Number theory
      • Arithmetic
      • Prime numbers
      • Fermat's last theorem
      • Cryptography

      Computing and information

      • Quantum computing
      • Complexity
      • Information theory
      • Artificial intelligence and machine learning
      • Algorithm

      Data and probability

      • Statistics
      • Probability and uncertainty
      • Randomness

      Abstract structures

      • Symmetry
      • Algebra and group theory
      • Vectors and matrices

      Physics

      • Fluid dynamics
      • Quantum physics
      • General relativity, gravity and black holes
      • Entropy and thermodynamics
      • String theory and quantum gravity

      Arts, humanities and sport

      • History and philosophy of mathematics
      • Art and Music
      • Language
      • Sport

      Logic, proof and strategy

      • Logic
      • Proof
      • Game theory

      Calculus and analysis

      • Differential equations
      • Calculus

      Towards applications

      • Mathematical modelling
      • Dynamical systems and Chaos

      Applications

      • Medicine and health
      • Epidemiology
      • Biology
      • Economics and finance
      • Engineering and architecture
      • Weather forecasting
      • Climate change

      Understanding of mathematics

      • Public understanding of mathematics
      • Education

      Get your maths quickly

      • Maths in a minute

      Main menu

    • Home
    • Articles
    • Collections
    • Podcasts
    • Maths in a minute
    • Puzzles
    • Videos
    • Topics and tags
    • Audiences

      • cat icon
        Curiosity
      • newspaper icon
        Media
      • graduation icon
        Education
      • briefcase icon
        Policy

      Secondary menu

    • My list
    • About Plus
    • Sponsors
    • Subscribe
    • Contact Us
    • Log in
    • Life on the beach with Markov chains

      Andrew Chambers
      29 November, 2021
      2 comments

      Markov chains are exceptionally useful tools for calculating probabilities and are used in fields such as economics, biology, gambling, computing (such as Google's search algorithm), marketing and many more. They can be used when the probability of a future event is only dependent on a current event. The Russian mathematician Andrei Markov was the first to work in this field (in the late 1800s).

      Beach life

      A Markov chain between home and the beach

      The picture above is an example of a situation which can be modelled using a Markov chain. We have two states: Beach (state 1) and Home (state 2). In our happy life we spend all the hours of the day in either one of these two states. If we are on the beach then the probability we still remain on the beach in one hour is 0.6 and the probability we go home in one hour is 0.4. If we are at home the probability we remain at home in one hour is 0.8 and the probability we go to the beach is 0.2.

      First we need to represent our information in a matrix form. A general 2×2 matrix is written as A=(a11a12a21a22) Where the subscript tells you the row and column (e.g. a12 tells you it is in the first row and 2nd column).

      For our Markov chain we define the following matrix: M=(p(m11)p(m12)p(m21)p(m22)). Here m11 is the situation of starting in state 1 and staying in state 1. And m12 is the situation of starting in state 1 and moving to state 2. The value p(m12) is the probability of starting in state 1 and moving to state 2. So for our beach existence we have: M=(0.60.40.20.8). The 0.6 shows that if we are already on the beach we have a 0.6 chance of still being on the beach in one hour.

      Where will we be in the future?

      For someone starting on the beach, the probability they'll be on the beach in two hours is the probability they stay on the beach that whole time, p(m11)p(m11), plus the probability that they go home and then come back to the beach, p(m12)p(m21). Here we see the benefit of Markov chains in that they allow us to utilise computer power to now calculate where someone will be in the future – simply by taking the power of the matrix. To find the all the probabilities after two hours I can square my matrix: M2=(p(m11)p(m12)p(m21)p(m22))(p(m11)p(m12)p(m21)p(m22)).

      Using the rules of matrix multiplication this then gives: M2=(p(m11)p(m11)+p(m12)p(m21)p(m11)p(m12)+p(m12)p(m22)p(m21)p(m11)+p(m22)p(m21)p(m21)p(m12)+p(m22)p(m22)). Here, in the top left entry, you'll recognise our probability of someone starting on the beach and being on the beach in two hours. We'll use arrow notation to represent the start and end state, so p(m1→1) means starting at state 1 (the beach) and ending at state 1 after two hours. We can then calculate all the probabilities as: M2=(p(m1→1)p(m1→2)p(m2→1)p(m2→2))=(0.440.560.280.72). We can see that for someone who started in the beach, the chance of them being on the beach in two hours is 0.44. Equally the probability of someone who started in the house being on the beach in two hours is 0.28.

      I can then carry on with matrix multiplication to work out where someone will likely be for any given number of hours in the future: Mn=(p(m1→1)p(m1→2)p(m2→1)p(m2→2)) where now p(m1→1) means starting at state 1 and ending at state 1 after n hours. So, for example, if I want to see where someone will be in 24 hours I simply do the matrix multiplications to get: M24=(p(m1→1)p(m1→2)p(m2→1)p(m2→2))=(0.3330.6670.3330.667). We can see that now it doesn't actually matter (to three significant figures anyway) where someone started – if they started on the beach there is a 0.333 chance they are on the beach in 24 hours, if they started in the house there is also a 0.333 chance they are on the beach in 24 hours. So I can conclude that as the number of hours increase towards infinity that the person in this scenario would spend 1/3 of their time on the beach and 2/3 of their time at home – not a bad life!

      A more demanding beach life

      A Markov chain between home, scuba and the beach

      We can see that things get much more complicated, even by adding just one extra state. Now we have 3 possible states: Beach (state 1), Home (state 2) and SCUBA (state 3). This time we need a 3×3 matrix: M=(a11a12a13a21a22a23a31a32a33) and as before we define our probability matrix for our example as: M=(0.60.10.30.10.80.10.50.10.4). For example the 0.8 in row 2, column 2 shows that there is a 0.8 chance of starting at home (state 2) and ending at home in one hour. Then, we can use matrix multiplication to calculate the probability of where we'd be after two hours (note we're using the same arrow notation as we did above): M2=(p(m1→1)p(m1→2)p(m1→3)p(m2→1)p(m2→2)p(m2→3)p(m3→1)p(m3→2)p(m3→3))=(0.520.170.310.190.660.150.510.170.32). This shows that after two hours there is (for example) a 0.19 chance that someone who started at home (state 2) is now found at the beach (state 1).

      After 24 hours we have the following probability matrix: M24=(p(m1→1)p(m1→2)p(m1→3)p(m2→1)p(m2→2)p(m2→3)p(m3→1)p(m3→2)p(m3→3))=(0.4070.3330.2600.4070.3330.2600.4070.3330.260). So we notice the same situation as last time – as the number of hours increase it gets less important where we started from. We can see that, to three significant figures, it doesn't now matter where we started – the probability after 24 hours of being found on the beach is 0.407, the probability of being found at home is 0.333 and the probability of being found diving is 0.260.

      Hopefully this is a quick example to demonstrate the power of Markov chains in working with probabilities. There is a lot more to explore!


      About the article

      Andrew Chambers has an MSc in mathematics and is a teacher at the British International School Phuket, Thailand. He has written news and features articles for the Guardian, Observer and Bangkok Post, and was previously shortlisted for the Guardian's International Development Journalist of the Year Awards. Combining his love of both writing and mathematics, his website ibmathsresources.com contains hundreds of ideas for mathematical investigations (and some code breaking challenges) for gifted and talented students.

      This article first appeared on ibmathsresources.com.

      • Log in or register to post comments

      Comments

      Prabhakar Rajasingham

      30 November 2021

      Permalink

      A lovely elegant exposition. Thank you!

      • Log in or register to post comments

      James B

      14 December 2021

      Permalink

      Nice demonstration, easily understood!

      • Log in or register to post comments

      Read more about...

      Markov process
      Markov chain
      probability
      matrix
      University of Cambridge logo

      Plus is part of the family of activities in the Millennium Mathematics Project.
      Copyright © 1997 - 2025. University of Cambridge. All rights reserved.

      Terms