The Waiting Time Paradox, Or, Why Is My Bus Always Late?

Nice article. It reminds me of my year living in London, and taking the bus everyday to Imperial College from West End Lane in West Hampstead. There was a stop on both sides of the road - one for the outbound bus, and one for the inbound (the bus went from central London to a terminus and then returned mostly on the same route). Now we did not use schedules - way too inaccurate at rush hour, and the busses there were pretty frequent anyway. But we did expect an even chance of the inbound bus arriving before an outbound one did. My daughter and I became convinced after a while that this was not happening, so we invented a game (which we called "The Game of Life".) When our bus (inbound) arrived first, we added 1 to our score. We subtracted 1 for every outbound bus that passed before ours arrived (there were often more than 1). We realized that the result would be slightly skewed to the negative, but we expected the outcome to be close to 0 over time. Of course it was not. Anyway we extended the game to many statistical situations. For example, you go to the checkout line at the supermarket, and there are N people in front of you. When you get to the front of the line, you count the people behind you - call that M. If M is bigger than N, you scored life points. If it is smaller, you lost some. So you add M-N to your running score, and you get an idea of how lucky you are in life. However, I never followed up with any real analysis, so I enjoyed this article.

This reminds me the bet in the bitcoin community [1]. If on average bitcoin blocks are produced every 10 minutes, and you learn that 5 minutes ago someone found a block, what is the average time you will wait for the next block? It turns out it&#x27;s 10 minutes, not 5 minutes as you would intuitively think. (it&#x27;s a memoryless process, so average expected time till block is always the same - 10 minutes - no matter how many blocks were recently found).<p>In other words, when you&#x27;re waiting for bitcoin transaction to be confirmed and go to check how long ago the most recent block was produced, in order to estimate how soon the next one will come - you&#x27;re doing it wrong. Even if previous block was found 9 minutes ago, you&#x27;re average waiting time for the next block is still 10 minutes.<p>[1]. <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;btc&#x2F;comments&#x2F;7rs8ko&#x2F;dr_craig_s_wright_has_refused_to_pay_up_on_a_bet&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;btc&#x2F;comments&#x2F;7rs8ko&#x2F;dr_craig_s_wrig...</a>

Slightly related, my ghost town had few buses and sparses. I could never rely on printed hours. If I got there 10 min earlier to be sure, I&#x27;d still never be sure I&#x27;d wait 20 min for nothing because it was 11 min early. Of course half the time if I decide to walk to the next town where buses are many, I&#x27;d see all my town buses (both ways) pass me &lt;yell-at-cloud.png&gt;<p>I think it made me completely careless about time, I would just go between stops and take the first one, go with the flow. By experience I&#x27;d know the range it would take for me to reach big places around the area.<p>I had a friend who was completely foreign to this mode of thinking, she was very dilligent and fully trusting (although she mostly used trains so a lot less divergence).<p>It reminds me of kid studies about intelligence &#x2F; wealth ratios. When you&#x27;re environment is random, you think random. When it&#x27;s predictable you planify.

&gt; a Poisson process is a memoryless process that assumes the probability of an arrival is entirely independent of the time since the previous arrival. In reality, a well-run bus system will have schedules deliberately structured to avoid this kind of behavior: buses don&#x27;t begin their routes at random times throughout the day, but rather begin their routes on a schedule chosen to best serve the transit-riding public.<p>I&#x27;ve never really understood any example involving a poisson process. They always seem to involve bus arrivals or light bulbs burning out, and I can&#x27;t understand why the memory less property would ever make any sense for these.<p>Even if the bus system was poorly run, why would it make sense to assume that the expected value of time to arrival doesn&#x27;t change based on how long you&#x27;ve been waiting?<p>What is an actual phenomenon that is well modeled by a poisson process?

Highly recommend reading this to any folks that are just sitting the discussions.<p>The simulations were worth the article on their own. The real world analysis was a great bonus.<p>Anecdotally, i was expecting confirmation bias to be the main culprit. Pleasantly surprised to seei was wrong.

Hah great analysis. One factor with bus&#x27; is the schedule is likely planned to minimize early arrivals at the risk of being late more often. Usually when a bus is early it has to sit and wait until its departure time. A late running bus can be more efficient, and if kept until departure time might not ever get a chance to average down the bursts of lateness.

OneBusAway is surprisingly accurate, at least in my experience. Google Maps has very good transit support too.<p>One reason buses are late is because a bus must travel a circuit. Cars provide linear transportation, so the delay can only happen in the direction of your travel. Since buses run a circuit, they are impacted by delays in the direction opposite of your travel as well.<p>Your bus might be late because the return route has traffic or other delays. Or maybe a drunk or drug user got in a fight with the driver and the police were needed. Or someone in a wheelchair had a problem getting onto the lift.

The memorylessness of the Poisson process makes the statistical aspect a bit trivial. But here&#x27;s an interesting variant: how should you update your beliefs while waiting if there is a certain probability that the bus won&#x27;t come at all? &quot;The Ups and the Downs of the Hope Function in a Fruitless Search&quot;, Falk et al 1994: <a href="https:&#x2F;&#x2F;www.gwern.net&#x2F;docs&#x2F;statistics&#x2F;bayes&#x2F;1994-falk" rel="nofollow">https:&#x2F;&#x2F;www.gwern.net&#x2F;docs&#x2F;statistics&#x2F;bayes&#x2F;1994-falk</a>

I&#x27;ve encountered the inspection paradox in debates about factory farming and people talking past each other points.<p>If you take the average farm, chances are that it&#x27;s doing humane farming. But if you take the average animal, it has an overwhelming chance of being in an industrial farm.

Reminds me of a similar article that measured a similar kind of question about the wait times for NYC subways conditional on how long you&#x27;ve been waiting (<a href="https:&#x2F;&#x2F;erikbern.com&#x2F;2016&#x2F;04&#x2F;04&#x2F;nyc-subway-math.html" rel="nofollow">https:&#x2F;&#x2F;erikbern.com&#x2F;2016&#x2F;04&#x2F;04&#x2F;nyc-subway-math.html</a>). I think it&#x27;s a pretty safe bet that people who like this post will like this article as well.

It strikes me that even with a perfectly regular starting schedule, buses might clump together in time because the schedule is probably dynamically unstable. To explain, picking up passengers from a stop costs time and a long time between buses implies a high probability that passengers will be waiting at a given stop. This further adding to the delay and shortens the time to the next bus in the schedule.<p>I&#x27;m sure drivers try to actively manage this, but if they didn&#x27;t I suspect the system would naturally evolve toward pairs of buses leapfrogging each other on long routes.

This Wikipedia article seems relevant: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Residual_time" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Residual_time</a><p>(From reddit - <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;programming&#x2F;comments&#x2F;9s4j58&#x2F;the_waiting_time_paradox_or_why_is_my_bus_always&#x2F;e8ntcng&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;programming&#x2F;comments&#x2F;9s4j58&#x2F;the_wai...</a> )

Can&#x27;t believe no one has yet mentioned the PASTA theorem - Poisson Arrivals See Time Averages (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Arrival_theorem#Theorem_for_arrivals_governed_by_a_Poisson_process" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Arrival_theorem#Theorem_for_ar...</a>). It is one of the theorems I remember the most from my Queuing Theory classes at the university!

In prague, the trams all run on time - within 2 minutes or less of the posted time. So I think this article is incorrect for this particular context.

Is the inspection paradox what would happen if you surveyed everyone on how many siblings they had, and every sibling double-counted N-1 times (where N is the number of siblings in their family), inflating the resulting &quot;average number of siblings&quot;, or is that something different?

On a related note, arrival time predictions can be biased early to prevent people from missing buses, which also increases the perception of lateness.<p><a href="https:&#x2F;&#x2F;nextbus.cubic.com&#x2F;FAQs" rel="nofollow">https:&#x2F;&#x2F;nextbus.cubic.com&#x2F;FAQs</a>

Is there a &quot;evil&quot; distribution which maximises the waiting time? Or is the Poisson distribution already the theoretical &quot;evil&quot; maximum that a public transport provider can achieve?

A bit off-topic: How can you integrate a jupyter notebook in a blog post like this one? It looks really nice!<p>Nice article, btw, interesting topic!

I hate the poisson distribution because it completely against the naive instincts of how random behaves.

Nice article. Since I just started learning Stats, I wish I could find more of such notebooks.<p>Any recommendations?

Would not be easier to actually time the actual waiting times as he waited for the bus every day?

This is a straightforward consequence of modelling an arrival process as a Poisson distribution with a constant rate of arrival lambda...<p>Go from arrival to cumulative arrivals to time of arrival to recurrence of arrival (next arrival). All are Poisson processes, including the recurrence process, which has a fixed expected value.

I’m glad I live in east Asia. Busses and trains are almost never late

It&#x27;s all much easier than that:<p>It&#x27;s just the Poisson process, e.g., with a nice chapter in E. Cinlar, <i>Introduction to Stochastic Processes</i>.<p>Buses come as <i>arrivals</i>. So bus arrivals are a <i>stochastic arrival process</i> where <i>stochastic</i> just means varying <i>randomly</i> over time where, really, the <i>randomly</i> doesn&#x27;t mean anything, includes deterministic arrivals, that is, known exactly in advance, but also admits any case of unpredictability.<p>Well, in short, if have a stochastic arrival process with <i>stationary, independent increments</i>, then the arrival process is a <i>Poisson</i> process and there is a number, usually denoted by lambda, so that the times between arrivals are independent, identically distributed random variables with exponential distribution with arrival parameter, the arrival rate, lambda. The <i>stationary</i> means that the probability distribution of the times between arrival does not change over time. The <i>independent increments</i> means that the time from one arrival to the next is independent of all the past <i>history</i> of arrivals.<p>The exponential distribution has the property, easy to verify with simple calculus, that the conditional expectation of the arrival time given that the arrival time is already greater than some number is the same as the expected arrival time.<p>So, net, if bus arrivals form a Poisson process, then the time until the next bus arrives is the same after waiting five minutes as not having waited at all.<p>Cinlar&#x27;s treatment is nice because it is <i>qualitative</i>, that is, has assumptions that can often be confirmed or believed just intuitively. And we might not believe that bus arrivals meed the assumptions.<p>This subject can continue with, say, <i>hazard curves</i> for equipment failures and a lot more about Poisson processes.<p>E.g., the sum of two independent Poisson processes, say, Red buses and Blue buses, assuming that they are Poisson processes, is also a Poisson process with arrival rate the sum of the Red and Blue arrival rates. If <i>randomly</i> throw away some arrivals, then what is left is also a Poisson process with arrival rate adjusted in the obvious way.<p>In Feller&#x27;s volume II is the renewal theorem that the sum of independent arrival processes, Poisson or not, with mild assumptions, converges to a Poisson process as the number of processes summed grows. So, if the users of a sufficiently busy Web site act independently with mild assumptions, then the Web site will see arrivals accurately as a Poisson process.<p>The vanilla Poisson process is Geiger counter clicks.<p>There is much more to the pure and applied math and applications of Poisson processes.

&gt; When waiting for a bus that comes on average every 10 minutes, your average waiting time will be 10 minutes.<p>This is very ambiguous. Unless he gives a time frame the numbers do not make sense. Average in a week? Average in a year? This is not how it works in real life.<p>And I cannot accept his premise. My experience tells me that, in New York, when I used to take a bus to work, sometimes the bus was coming as I was walking to the stop; sometimes I would wait a long time. Sometimes not very long. There was no observable bias.