Modes, Medians and Means: A Unifying Perspective

There is a much wider generalization here which is studied under the name of &quot;property elicitation&quot; in computer science, machine learning, and statistics.<p>The generic question is: Given a loss function, what &quot;property&quot; of the distribution minimizes average loss; and given a &quot;property&quot;, characterize all such loss functions. For example, Bregman divergences are (essentially) all losses that &quot;elicit&quot; the mean of a distribution. If you have any monotone continuous function g(), then |g(x) - g(s)| actually also elicits the median, and these are essentially all that do.<p>Apologies for self-promotion, but you can read more at references on this page (disclaimer: I&#x27;m one of the researchers who posted it): <a href="https:&#x2F;&#x2F;sites.google.com&#x2F;site&#x2F;informationelicitation&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sites.google.com&#x2F;site&#x2F;informationelicitation&#x2F;</a><p>or tutorials on this subject at my blog: <a href="http:&#x2F;&#x2F;bowaggoner.com&#x2F;blog&#x2F;series.html#convexity-elicitation" rel="nofollow">http:&#x2F;&#x2F;bowaggoner.com&#x2F;blog&#x2F;series.html#convexity-elicitation</a>

One small extension:<p>* L_0 -&gt; mode<p>* L_1 -&gt; median<p>* L_2 -&gt; mean<p>* L_infinity -&gt; midrange, I think<p>that is, (smallest observation + largest observation)&#x2F;2<p>(BTW, the author is also a big contributor to the wonderful Julia language, I believe)

I should note that square-error loss is not the only one that gives the average. Log-loss is another one, for example (you can prove it by taking the derivative of your minimization, setting it to zero and solving)

These relationships are pretty clear once you see the other distributional metrics. Going further there&#x27;s skewness and kurtosis for the third and fourth statistical moments, resp.

there is so much unexplained beauty in this text!<p>notice that the unifying perspective depends on a continuous parameter p, which is 2 for the mean, 1 for the median and -infinty for the mode. Thus, there is a continuous family of statistics that interpolate between these three things!<p>he does not mention it neither, but this means that you can define modes of a continuous variable (without resorting to histogram bins)

For mode to fit into this unifying framework, this article assumes that 0^0 is not indeterminate and is instead simply 0 (instead of the more usual assumption of 1).<p><a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Zero_to_the_power_of_zero" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Zero_to_the_power_of_zero</a><p><a href="http:&#x2F;&#x2F;mathforum.org&#x2F;dr.math&#x2F;faq&#x2F;faq.0.to.0.power.html" rel="nofollow">http:&#x2F;&#x2F;mathforum.org&#x2F;dr.math&#x2F;faq&#x2F;faq.0.to.0.power.html</a>

Mods, could you maybe put (2013) in the title? Content is timeless, but just as a heads up that one might have come across it before.

I remember introducing someone to the concept of variance in a set of data and I used a very similar approach. Variance seems like an arbitrary (but obvious) definition but in fact it can be derived from first principles by just looking for the simplest possible function that firstly has some dependency on the difference between values and the arithmetic mean, secondly has the property that it is independent of whether the differences are positive or negative (to the right or left of the mean) and thirdly that it does not depend on the size of the data set (i.e. duplicating each member of a data set would leave the variance unaffected). When you consider each of these, the equation for variance arises very naturally.<p>Arithmetic difference satisfies for first property<p>Squaring each difference satisfies the second property<p>Taking the arithmetic mean satisfies the third property<p>Var(X) = E[(X - mean)^2]

While I loved this post, could have also been helpful to include more info about Pythagorean Means (geometric mean and harmonic mean). Worth looking into depending on the type of variability seen in your data.<p>EDIT: just saw this was mentioned in one of the comments...

The Generalized mean[1] linked to in the blog comments was similarly insightful.<p>It unifies the inequalities:<p>max &gt; root mean square &gt; arithmetic mean &gt; geometric mean &gt; harmonic mean &gt; min<p>that I remember from highschool math competitions.[2]<p>[1] <a href="http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Power_mean#Special_cases" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Power_mean#Special_cases</a><p>[2] <a href="https:&#x2F;&#x2F;artofproblemsolving.com&#x2F;wiki&#x2F;index.php?title=Root-Mean_Square-Arithmetic_Mean-Geometric_Mean-Harmonic_mean_Inequality" rel="nofollow">https:&#x2F;&#x2F;artofproblemsolving.com&#x2F;wiki&#x2F;index.php?title=Root-Me...</a>

Is there any fundamental reason to measure discrepancy by abs(s - x_i)^2 rather than say by abs(s - x_i)^1.5? Is something special about 2 in this context, or is it just a social convention that seems to work pretty well?

If your loss function is an actual financial $ loss (or revenue), then arithmetic-mean times n gives the best estimate of total&#x2F;long-run expected loss.<p>If the distribution of losses is skewed or has outliers then estimates other than the mean (median, trimmed means etc) often under-estimate total losses.<p>Under-estimating total losses in the long run could be very bad for business.

In case anyone wants to dig into the follow up on Lp norms, here it is: <a href="http:&#x2F;&#x2F;www.johnmyleswhite.com&#x2F;notebook&#x2F;2013&#x2F;03&#x2F;22&#x2F;using-norms-to-understand-linear-regression&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.johnmyleswhite.com&#x2F;notebook&#x2F;2013&#x2F;03&#x2F;22&#x2F;using-norm...</a>

Painfully, American families are learning the difference between median and mean<p><a href="https:&#x2F;&#x2F;qz.com&#x2F;260269&#x2F;painfully-american-families-are-learning-the-difference-between-median-and-mean&#x2F;" rel="nofollow">https:&#x2F;&#x2F;qz.com&#x2F;260269&#x2F;painfully-american-families-are-learni...</a>

The beauty of math is often missed. That&#x27;s why we say maths is an art!

Is there similar generalization for geometric mean and geometric median?

&quot;To sum up, we’ve just seen that the three most famous single number summaries of a data set are very closely related: they all minimize the average discrepancy between s<p>and the numbers being summarized. They only differ in the type of discrepancy being considered:<p><pre><code> The mode minimizes the number of times that one of the numbers in our summarized list is not equal to the summary that we use. The median minimizes the average distance between each number and our summary. The mean minimizes the average squared distance between each number and our summary.&quot;</code></pre>

Math does not render unless I allow cloudflare.com to execute scripts. Why can&#x27;t we just self-host scripts, is it that hard?

On my phone, this entire article reads “blah blah blah [Math processing error] blah blah blah [Math processing error] blah blah blah [Math processing error] blah blah blah [Math processing error]”

To get the median of an even number of values, you must calculate the mean of the middle two values. Therefore the definition of the median relies on the mean already being defined when working with a discrete number of values, which isn&#x27;t really explained in the post.<p>In fact, there&#x27;s a whole spectrum of averages defined with mean and median on each end, depending on how many outliers you eliminate. For example, if you have eight numbers, you can define a spectrum of four averages:<p><pre><code> 2,3,5,7,11,13,17,19 &#x2F;&#x2F; mean, here 9.6250 3,5,7,11,13,17 &#x2F;&#x2F; mean with outlier on each side stripped, here 9.3333 5,7,11,13 &#x2F;&#x2F; mean of central two quartiles, here 9.0000 7,11 &#x2F;&#x2F; median (i.e. mean of center two numbers), here 9.0000 </code></pre> You could then repeat the process on that spectrum of averages to get a shorter spectrum, here [9.2396 (mean), 9.1667 (median)], recursively until you have one &quot;mean-median&quot; left, here 9.2031.<p>I wonder how this fits in with the explanation in the post.