We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results.<p>So he replaced the model with a single line of code with one boolean expression made with those 4 parameters connected with logical operators.
I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot."<p>there's really no other magic in it than that.
Why can’t there be a discussion on machine learning without everyone on HN trying to prove how unnecessary it is. Queue the anecdotes on simpler regression based methods, over paid scientists, and how much superior some other simpler method is.
Is linear regression programming?<p>A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures).<p>Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network.
I would go a step further, and say that prompt design will become an important sector of programming.<p>Modern language models (eg GPT-3 et al) offer the capability to take a natural language input, match it against the context of the sentence, then propose a query that is understandable to the layperson. This abstraction allows us to understand the problem better, rather than just analyzing the way the problem manifests itself in code. Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available.<p>The next wave of programmers will need to understand how human language can be used to efficiently guide models to solve problems that can’t be solved by human-written code. This is a big challenge, and we’re just at the very beginning of it, but I think it will open up new and undiscovered ways to create value in the world.<p>I have quite a few additional thoughts on the topic which I’ve captured here: <a href="https://sundayscaries.substack.com/p/whos-the-real-expert" rel="nofollow">https://sundayscaries.substack.com/p/whos-the-real-expert</a>
Over time the words 'teaching' and 'programming' will converge and at some point you likely won't be able to tell the difference between the two anymore (in a computer context).<p>Deep learning isn't programming per-se, but it definitely creates results that are of the same kind that programming would be able to create as well (in principle, at least, in many cases).
As I've understood it, normal programming is transforming an input with a program to get an output.<p>Machine learning is giving the input and the output to get a program.<p>Problem is, it's too difficult to summarize or understand the resulting program, while the program you get is tied to the output data which is never really accurate.<p>I'm still curious how ML specialists are approaching the task of analyzing a resulting deep neural network, and squeeze some science from it (meaning putting words on things they understand and are able to explain).<p>I've also read that google was using ML to test different learning models, to easily find the best model to use for a given problem. I'm not sure but it sounded like they were feeding the training model and the data into another learning model. I can't remember the details or the article or the reddit comment but it sounded quite interesting.
for anyone interested, chris olah has a good blog post providing further insight on this topic: <a href="https://colah.github.io/posts/2015-09-NN-Types-FP/" rel="nofollow">https://colah.github.io/posts/2015-09-NN-Types-FP/</a>
I wouldn't say that deep learning is programming. I think the key feature of programming is <i>legibility</i>. A program is something that is clear enough to read and understand, to be decomposed in its constituent parts, and that has understandable semantics.<p>For example, writing an algorithm that has precise steps and procedures is programming. Putting my input into a box, shaking the box, and taking the result out is not programming, even if the box somehow solved the problem. Merely describing a problem and then having it solved is not enough to delineate programming, because that actually does apply to almost anything.
Somehow remind me of Software 2.0[1]<p>[1] <a href="https://medium.com/@karpathy/software-2-0-a64152b37c35" rel="nofollow">https://medium.com/@karpathy/software-2-0-a64152b37c35</a>
Having thought along these lines before, I realized that i didn't get any new useful insights by throwing neutral networks and conventional programming into the same bucket. Life went on as usual in both the worlds and I stopped thinking about it.<p>Any insights worth learning about?