Mathematicians becoming data scientists

I was a PhD student in mathematical logic, and I know a couple people who got PhDs in this area and became data scientists. One thing about this field (and other areas of math) is that almost everyone in it feels mediocre. It's like 10% of the people are 10x better than the other 90%, 1% are 10x better than the next 9%, .1% are 10x better than the next .9%, etc. The top two people in the field seem to be notably better than everyone else. Another thing is that the problems are incredibly arcane. They have no connection to practical concerns or even often other parts of math, and also you can't really explain what you are working on to someone who works in a different area of mathematical logic (unless maybe the person is extremely good). So you have a lot of people getting PhDs who are very, very smart by any normal measure but who would be doing mediocre and arcane work in mathematical logic. Part of the appeal of working in data science or working for the NSA or something is that it is somewhat down-to-earth and people will think you are really smart even if you are at the bottom of the top .01%. (And you can do really great work at this level -- I don't mean to say it is all about caring what other people think.) It is a little weird that the blog post is so negative in a way.

One side of this that bugs the crap out of me: I have a PhD in math and, after getting sick of teaching mediocre students with a palpable aversion to mathematics, I spent a couple of years applying for data science jobs and getting very few responses. The few interviews I got seemed to go well, but I had no offers and gave up on leaving academics. I have experience in software development (C++), I have a portfolio of statistics/data analysis projects in R/RStudio, but what I think is toxic is having spent so many years in academics; I finished my Phd in 2003. Being an over-40 academic feels like an insurmountable liability, even though I have many of the skills that people complain about being in short supply. And another thing: get off my lawn, you rowdy kids! :|

I had two blog posts exactly on that topic, math&#x2F;phys -&gt; data science:<p>- why: <a href="http:&#x2F;&#x2F;p.migdal.pl&#x2F;2015&#x2F;12&#x2F;14&#x2F;sci-to-data-sci.html" rel="nofollow">http:&#x2F;&#x2F;p.migdal.pl&#x2F;2015&#x2F;12&#x2F;14&#x2F;sci-to-data-sci.html</a> (on academia vs industry)<p>- how: <a href="http:&#x2F;&#x2F;p.migdal.pl&#x2F;2016&#x2F;03&#x2F;15&#x2F;data-science-intro-for-math-phys-background.html" rel="nofollow">http:&#x2F;&#x2F;p.migdal.pl&#x2F;2016&#x2F;03&#x2F;15&#x2F;data-science-intro-for-math-ph...</a> (also got reprinted at KDnuggets)

Slightly off topic: They still can&#x27;t build proper software. Academics (including mathematicians) are notoriously bad at writhing production grade software. This leads to handovers of &#x27;proof of concepts&#x27; to seasoned software developer team who than struggle with the (often complex) mathematics&#x2F;science behind it. Imho universities should give a bit more attention on how to write quality software; a bit of test driven development and continuous integration is not that hard and would massively improve the quality of the software written by scientists.

I interview probably two to three people a week for a DS job. It&#x27;s a really difficult role to hire for. I&#x27;m basically looking for mathematicians with some knowledge of stats who also enjoy coding and follow best practices in coding (i.e. They didn&#x27;t just pick it up and hack something that works but care enough to document and structure so others can follow it at a minimum) but we also need the MBA component as well, a large part of this industry is how do you take what a DS does and deploy it in a business and ensure the business derives value from it. But I think coming at it from a mathematical background puts you in a much stronger footing than the cs guys who try to learn the math later. I rarely find strong candidates without a formal mathematical background.

What about the other way around?<p>I&#x27;m from an engineering background, going in the direction of data scientist. Sometimes I find that my math skills could be stronger, and I try to read up on things when I encounter them, but still it sometimes feels like there is an infinite amount to learn. Maybe I could use some more systematic approach to it. Anyone else who has walked this path, and could come with some useful advice&#x2F;resources?

An earnest question I&#x27;ve heard from people trying to switch is: what&#x27;s a good benchmark for testing where you stand in terms of coding skills? How do you calibrate and measure your production-quality coding skills in Scala, C, etc. when you have spent all your life in academia?<p>I feel Kaggle is sufficient for the exploratory part of data science. But Kaggle&#x27;s relevance to testing and honing &quot;production quality&quot; code-writing skills is sometimes minimal.<p>In the larger context of programming and software engineering, scientific programming is fairly easy to code-up (though they are harder to conceptualize and understand mathematically). The coding part of non-CS&#x2F;non-CE&#x2F;non-some-parts-of-EE academia is pretty much mostly scientific programming. Yet, production quality code is seldom just scientific programming.

Note: this is the blog of Jordan Ellenberg (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Jordan_Ellenberg" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Jordan_Ellenberg</a>).

This post mostly describes me. I am a PhD student in applied mathematics and returned school to get my PhD precisely because I wanted to get a job outside of academics, most likely as a data scientist or scientific programmer. 2.5 years into my program I still struggle to see beyond the degree but am currently looking at non-academic internship opportunities as a way to start to untangle myself from academics. I also spend a significant amount of my time trying to teach my self good programming practices that I don&#x27;t get from mathematics (how to write clean, adaptable code, choosing appropriate design patterns and writing generic code without over-abstracting, writing readable documentation and using version control, etc.) To those of you who have left academics do you have any other suggestions? How about for marketing myself as a programming mathematician when competing against CS and engineering students for internships&#x2F;jobs?

&gt; Do you like modeling complex problems mathematically?<p>Do you enjoy kicking the ball towards the gate and scoring? Then you might be good for Football.

I&#x27;ve been encouraging my daughter, a statistics major, to pursue data science by including Python&#x2F;R in her studies and then possibly heading back for an MBA. But not sure if an MBA would be a benefit. Thoughts from actual data scientists?

As a mathematician who successfully made the leap, I think the biggest requirement is ability to swallow your pride and &quot;lower&quot; yourself to seriously study professional coding.<p>There&#x27;s too much temptation to hack something in Python or whatever, google as you go, etc. What I did was sit down and practically memorize entire programming manuals. Many academics would refuse to do something so plebeian.

I remember asking why i was rejected from Palantir and vaguely recall the phrase &quot;too mathematical&quot;.<p>After years of trying I have been unable to make the transition from Mathematics into Data Science and have since shifted my attention to other things.

If you see programming as a tool for solving domain-specific problems, then domain experts who can exploit the tool obviously have an edge over programmers who rely on inputs from these experts when developing domain-specific applications.

I guess it is easier to get into data science as a math graduate. I&#x27;m trying without a degree, and haven&#x27;t succeeded yet.

A few fellow students graduated with a BA and had their aplications rejected by most companies due to a lack of experience.<p>Theory is nice. But Experience trumps all. Esp when you think it mostly boils down to PCA&#x2F;DBSCAN, regression and scatter matrices.<p>A month of Python&#x2F;R&#x2F;D3 on some initial data like pubmed&#x2F;twitter or any toy dataset would go a looong way.

&gt;. A big thing about the transition to tech is that you possibly start communicating with people who don’t really know what a vector space is. Be ready to have those conversations. Honestly ask yourself if you’re OK with having those conversations.<p>This is so snobbish. What here is even think about ?

There is positive correlation between applied math and programming. I see expertise in these areas coupled with a knack of getting a point across as a potential career opportunity. No wonder academics in some parts of Asia have, for generations, seen math as “evergreen.”

A lot of this can also generally apply to a Computer Science Ph.D graduates who would like to understand what working in the industry entails.

I think MBA adds more value to your career in globalization

<i>•Can you walk away from a problem when the solution is “good enough,” are you able to switch between tasks or problems with relative ease? Are you OK with simple solutions to problems that could have more complicated solutions, but only with rapidly diminishing returns?</i><p>Translation: Can you be a good assembly line worker, devoid of pride or curiosity and just do the damned job? If so, great, we can use you! If not, please return to academia.<p>Lovely stuff.