Guide to Linear Regression (2015)

Any summary of linear regression ought to at least point out that the &quot;cost function&quot;, which depending on the circumstances is often viewed as modeling the variance in the data, is typically limited to one data coordinate. That is, the separations to the fit line to be minimized go entirely along the y-axis; which has the effect of assuming that there&#x27;s perfect knowledge of the x value. And while that can be the case for some sampling protocols, it is also often not the case. So please consider including something like:<p>If there is uncertainty in both the x and y coordinates then one needs to pursue alternate approaches which admit variation in both, one of the most popular being &quot;Deming regression&quot;[1]<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Deming_regression" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Deming_regression</a>

I believe Frank Harrell&#x27;s Regression Modeling Strategies is one of the best references for performing regression analysis.<p>website: <a href="http:&#x2F;&#x2F;biostat.mc.vanderbilt.edu&#x2F;wiki&#x2F;Main&#x2F;RmS" rel="nofollow">http:&#x2F;&#x2F;biostat.mc.vanderbilt.edu&#x2F;wiki&#x2F;Main&#x2F;RmS</a> Amazon book: <a href="http:&#x2F;&#x2F;amzn.to&#x2F;1UdPWOv" rel="nofollow">http:&#x2F;&#x2F;amzn.to&#x2F;1UdPWOv</a>

Yikes, no discussion of how one might interpret this model, what its assumptions are, and how to assess validity or fit? Look elsewhere--this is not the guide to linear regression anyone should be looking for.

Multivariate statistics is something I really wish was covered better.<p>In my work as an engineer I use multiple linear regression from time to time. The best method I&#x27;ve found is forward selection stepwise approach. I&#x27;ve never really seen a simple explanation of how it works (it was introduced to me by a statistician at my work) but it is useful when you have many variables and you want to see how significant a single variable is relative to overall regression. The impression I get is pure stats people really dislike stepwise modelling.<p>More recently I&#x27;ve been looking into something called PCA (Principle Component Analysis) and PLS (projection to latent structure) after I came across it in a PHD thesis. I&#x27;ve yet to find a decent simple explanation about how it works though. Unfortunately my work no longer employees a statistician.

IMHO, a very confusing thing about Linear Regression is that there&#x27;s widespread disagreement about whether the data are supposed to be Normally Distributed or not, and on how to measure said Normality.