markov-chains-gagniuc

Python implementation for the supporting algorithms from the book Markov Chains: From Theory to Implementation and Experimentation by Paul A. Gagniuc

Overview

The book contains 17 supporting algorithms in 3 languages Visual Basic, PHP and Javascript:

1. The algorithm simulates a sequence of draws from 2 jars with black and white balls. The ball drawn indicates the color of the jar used for the next draw;
2. The algorithm extends the previous one with a function that builds the jars with different ball ratios;
3. The algorithm extracts transition probabilities from a sequence of events;
4. The algorithm generates predictions from transition probabilities;
5. The algorithm generates predictions until a steady state is detected;
6. The algorithm calculates the steady state using a closed-form equation;
7. The algorithm combines the extraction of probabilities with the generation of predictions;
8. The algorithm extends the previous one to work with 3-state chains;
9. The algorithm extends the previous one to work with 4-state chains;
10. The algorithm is presented in the book with improvements to allow more states dynamically;
11. The algorithm converts integer values into discrete bands;
12. The algorithm extends the 3-state version with an absorbing state;
13. The algorithm presents the 4-state version as a configurable n-state version;
14. The algorithm builds a 3-state machine with jars from given ball ratios;
15. The algorithm extends the previous one with a 4-state machine;
16. The algorithm calculates probabilities to validate previous version;
17. The algorithm uses a 4-state machine to calculate the average time spent in each state;