Markov chains are super useful in statistics but it isn't obvious at first what problem they solve and how - some further reading that I found helpful:<p><a href="https://twiecki.io/blog/2015/11/10/mcmc-sampling/" rel="nofollow">https://twiecki.io/blog/2015/11/10/mcmc-sampling/</a><p>Note that the point of the markov chain is it's possible to compute <i>relative</i> probabilities between two given points in the posterior even when you don't have a closed form expression for the posterior.<p>Also, the reason behind separating the proposal distribution and the acceptance probability is that it's a convenient method to make the Markov process stationary, which isn't true in general. (Wikipedia page on MCMC is also useful here).
This is timely! I have an assignment on these coming up soon. Can anyone with knowledge about this explain something. From what I can tell, many matrix multiplications move vectors so they are more inline with eigenvectors if they exist. So Markov Chains are just a continual movement in this direction. Some examples that don't do this that I can think of are the Identity matrix and rotations.. Is there a way to test if a matrix will have this effect? Is it just testing for existence of eigenvectors?
The relevance to me is that markov chains are a remarkable way to explain why LLMs are both useful and very unreliable.<p>You train on piece of text and then the output 'sounds' like that text it was trained despite being pure gibberish.
markov chains are used for my favourite financial algorithm; the allocation of overhead costs in cost accounting. wish there was an easy way to visualise a model with 500 nodes