Ask HN: Why hasn't the deep learning community embraced Julia yet?

42 pointsby newsoulover 2 years ago

Julia is a language created by academicians. Unlike Python, it is less headachy. Python has weird behaviours scattered all over the place. Still everyone uses Python for Deep Learning (including researchers).Julia is fast, with good syntax and lots of libraries. Infact it has a library called Flux.jl for ML/DL. Yet nobody uses it. Why? No major industry partner has picked it up to create bigger deep learning frameworks.Why?

32 comments

runeblazeover 2 years ago

Julia is too quirky. Its one-indexing thing, general time-to-first-plot problems, and its multiple dispatch by default makes it hard to advocate to a general audience (or a community as big as "Deep Learning community").The ecosystem also feels weird. For example for CLI parsing in Python I do argparse/click, in Rust I do clap. Both work well (Rust especially so). Julia seems to have ArgParse.jl which I found unpleasant. My guess is that this sentiment spills to other areas as well in the ecosystem. I am sure that Julia has very well designed libraries, but most of the rest are not so honed for practical use, and most importantly, probably lacks some features that Python-equivalent ones have.Also personally (I am far from a DL researcher but I do research), I find Julia's design headachy as well quite some times. I like Python yield statements for iterators but the equivalent in Julia is always more complicated. Making Julia code performant requires another layer of intuition of how the JIT works. Julia has type annotations but "type annotations as lightweight formal methods" often cannot be fulfilled due to design (for example one cannot annotate the equivalent of `map :: (a -> b) -> [a] -> [b]`). Of course I stand corrected as only a casual Julia user.I also want to say that I only stated my criticism towards the language. There are many many good things about Julia that I will just omit due to time.

评论 #32806820 未加载

mxkopyover 2 years ago

Julia was made by academicians, and there's a reason we distinguish between academia and industry.In academia code is the means to an end, which is usually a paper. This means that the experience of writing the code, making sure it's portable, readable, etc. don't really matter. These are all industry concerns; specifics of implementation, not algorithm design/mathematical questions.This isn't reflected in Julia as a language as much as it is in Julia as an ecosystem. Some of the packages I work with are poorly maintained or designed with 0 thought to the issues I described earlier. And since Julia is so new, there are few alternatives.Imagine if your software was written by your college professor who'd pushed to prod maybe like 3 times in their life. Instead of just fixing the damn bug they're liable to make you fix it instead as an 'exercise for the reader', or posit that the problem input is outside of the scope of their class and refer you to another one.My example of this is the Colors.jl package. Images, rather than being 3D arrays of floats/ints, instead become 2D arrays of RGB/HSV. But you can't do any arithmetic on RGB/HSV values, you need ColorVectorSpace.jl for that. I'm sure it makes sense, but I can't help but feel as if sense is taking precedence over the programmer's experience in many of these packages.Again, I don't think Julia as a language is unfit for widespread use. But before it can be, it needs to burst out of its ivory bubble.

评论 #32848070 未加载

oxffover 2 years ago

You could reword the question to be about Rust and C/C++ and much of the same would apply.There are generic answers:- Refactoring business logic costs. That's why it doesn't happen often.- Interop between two different codebases (new Julia, old Python) costs (both at an org scale and at programming scale).- It's a new lang. Hiring new programmers to a new meme language is very risky. You lose that programmer, your project is in trouble.- You are underrating Python ecosystem strength.- Python first mover advantage.There are Julia specific reasons:- No Julia programmers outside of MIT where as everyone has touched Python.- Python is "simpler".- Last I read Julia has some growing pains, like REPL boot times?

评论 #32806540 未加载

评论 #32807702 未加载

评论 #32806362 未加载

jamestimminsover 2 years ago

As someone once said, "Python isn't the best language for anything, but it's the second-best language for almost everything".If we don't get hung up on the exact literalness of that statement, there's a lot of truth to it. Being essentially a "lingua franca" that's easy to pick up has accelerated it in various seemingly unrelated fields. This, I assume, it to the seeming detriment of use-case specific languages like Julia.

patrickkidgerover 2 years ago

There's lots of good reasons. Most notably the autodiff in Julia is unreliable -- there's been quite the litany of packages trying/failing to do this robustly and efficiently. (This issue pretty much kills its possibilities straight from the get-go.)The fact that the ecosystem is largely created by academics is also a serious issue. Code quality is very low / things silently break / etc. This is partly an issue with Julia itself; the language offers few tools to verify the correctness of your code. (E.g. interfaces etc.)Julia vs XYZ is an old topic. And yet this keeps coming up! There's quite a lot of blog posts on the topic. (E.g. here's mine: <a href="https://kidger.site/thoughts/jax-vs-julia/" rel="nofollow">https://kidger.site/thoughts/jax-vs-julia/</a>)

评论 #32807951 未加载

chadykamarover 2 years ago

Julia's main selling point over Python is its speed, but most Python deep learning libraries are just wrappers around C/C++/Fortran/CUDA/Cython codebases that are as fast or faster than Julia.

评论 #32806700 未加载

Teodolfoover 2 years ago

Julia provides no meaningful advantages. PyTorch and JAX are too good. For typical deep learning workloads, Julia will not easily have a speed advantage. Everything goes down to the same cuDNN kernels anyway.Julia seems like an attempt at a better matlab, but the machine learning world moved to python first.(Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)

评论 #32806597 未加载

评论 #32806919 未加载

curuinorover 2 years ago

'language created by academicians' is like, an anti-selling point (in fact, python was created when von rossum was in grad school so it's still created by academicians, just got lots of development in the 30 years after)julia also has weird behaviors scattered all over the place

评论 #32807452 未加载

ansgriover 2 years ago

As a former academician, “created by academicians” is for software as bad an advertisement as it gets, especially for something foundational and impossible to change later.I don’t think it’s a meaningful factor in case of Julia though: more like that Python is a safe and rather pleasant choice for most research-related problems, not only in ML. So no reason to change. And if you’re just starting, you’d better learn Python anyway, and if it’s enough, why look for something marginally better?Julia needs a niche where it’s clearly better than Python (and R and C++ for that matter) and grow from there. Maybe there already is one, they need to market it more clearly.

TekMolover 2 years ago

I might be not the average user, but then again users are usually very heterogeneous. So here is my Julia journey:When I want to experience a new language, I got to a place on the web, where I can try it.For Python, there are <a href="https://www.online-python.com/" rel="nofollow">https://www.online-python.com/</a> and many others.For PHP there are <a href="https://3v4l.org" rel="nofollow">https://3v4l.org</a> and many others.For Javascript there are <a href="https://jsfiddle.net/" rel="nofollow">https://jsfiddle.net/</a> and many others.But every time I think "Hey, I might give Julia a try", I go to Google and type "try julia online" and find nothing. Every site either wants me to sign up or is dysfunctional. So I think "What? That cannot be. Lets try a different search". And search for "julia online repl". Again. Nothing works. And I give up.I just did this again.And I might try again in a year or so.That's the situation here regarding Julia.

评论 #32806662 未加载

评论 #32806625 未加载

ur-whaleover 2 years ago

. late to the game. slow startup time. 1-based indexing. more complicated than python (many ML practitioners aren't really great at CS). native speed (a clear Julia win) doesn't really matter in the ML world: in ML, speed matters on learning and inference, where the language used to build and massage models is irrelevant, all of them compile models to run on GPU/TPU/etc ...

tharneover 2 years ago

When folks wonder why Julia hasn't displaced Python or R, despite being better on a number of fronts, I'm always reminded of this quote (courtesy of wikipedia) by Eric Raymond around why plan9 never went on to displace Unix:> [I]t looks like Plan 9 failed simply because it fell short of being a compelling enough improvement on Unix to displace its ancestor. Compared to Plan 9, Unix creaks and clanks and has obvious rust spots, but it gets the job done well enough to hold its position. There is a lesson here for ambitious system architects: the most dangerous enemy of a better solution is an existing codebase that is just good enough.Python is good enough, and Julia isn't "Better enough".

sam0x17over 2 years ago

Everyone forgets the obvious fact that python is a default install on most linux distros and many, MANY researchers are in government labs or universities where installing software is tightly controlled and are presented with choosing between waiting months to get approval to install something versus starting coding now today in python.. I swear Python rose to popularity in the DL community for this very reason -- for many researchers it was all they could use without a lengthy approval process.That said I'm no lover of python, I just think this is a huge contributing factor to its rise in popularity. A little conspiracy theory, if you will ;)

评论 #32806828 未加载

liminal18over 2 years ago

Note: not an expert or data scientist this could be an uninformed opinion.R has multithread support: <a href="https://josephcrispell.github.io/2018/08/27/multi-threading-R.html" rel="nofollow">https://josephcrispell.github.io/2018/08/27/multi-threading-...</a>Julia's main draw was native really fast code with multi-thread/parallel support.The other issue is that a parallelized c version of a data science algorithm can be scripted by Python or R pretty easily.

k4ch0wover 2 years ago

I think jupyter notebooks, pytorch/tensorflow and python is much more approachable by everyone. Julia has a steeper learning curve whereas I think you can learn to do enough in Python within a month to be dangerous.

jgwernerover 2 years ago

We (<a href="https://illumidesk.com" rel="nofollow">https://illumidesk.com</a>) support Julia notebooks (leveraging Jupyter Server Kernels using a variety of frontends, such as JupyterLab, VS Code, etc). The Robotics department (taught by Professor Jessy Grizzle at the University of Michigan) uses Julia extensively. Over the course of the last three years we have learned a few things about Julia when compared to Python and friends:- Julia is faster at reading CSV's compared to using something like Pandas (presumably because it's multi-threaded and Python is single threaded) - Julia does seem faster with read operations, but it's important to write code that declares types (myIntArray = Int[] vs myIntArray = []), this way Julia would read from a contiguous block of memory instead of reading an array of pointers in memory - Julia's packages are limited to Python's equivalentsOur take is that Julia is amazing for ML/DL but Python is better for entry level data scientists. Once the data scientist gets up to speed with Python then Julia is a nice feather in the cap for more specialized use cases.

empikoover 2 years ago

I am genuinely curious. Does it have anything comparable to the following?- pytorch or tensorflow - pandas - numpy - scipy - matplotlib - huggingface transformers - jupyter

评论 #32806447 未加载

评论 #32806475 未加载

评论 #32806430 未加载

评论 #32806423 未加载

rg111over 2 years ago

You can read this excellent piece [0] written from experience.Julia is still far from ready for serious adoptation for heavy workflows.For libraries, the optimization cannot be trusted. And it cannot be said many times that best practices were followed.While you can trust NumPy or PyTorch, you cannot trust a Julia library at the same level.[0]: <a href="https://yuri.is/not-julia/" rel="nofollow">https://yuri.is/not-julia/</a>

jasfiover 2 years ago

There are just too many new languages all vying to be the new Python. I personally like Nim, but it's also gaining traction too slowly.

lizen_oneover 2 years ago

I used Julia in a robotics project doing statistics/estimation/easy optimization but not deep learning. I also do ML/DL:Julia vs. Python- PyTorch is standard and it is hard to convince other people to switch- long compile time on startup during deployment (not so good for a robot) but also for plotting; other people really hated thisJulia vs. C++- Julia has a JIT and is MUCH faster than Python if you cannot write it as a sequence of numpy operations, e.g. if you have loops and if-blocks in the main loop; C++ obviously also shines here- however, similar to Python you can only detect problems of the code when running it - the linters etc. are not good enough; hence, I also fear changing only a few lines; programming in C++ is much easier and you have much more confidence that the code is correct if it compilesAfter learning JAX in Python, which compiles numeric code JIT, I have almost no reason using Julia anymore. Of course, DifferentialEquations.jl and many optimization libraries are top notch.

xyprotoover 2 years ago

To me, it's because it doesn't feel reliable by default. For example:Python:<pre><code> >>> 2**2**2**2**2 Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: Exceeds the limit (4300) for integer string conversion </code></pre> C:<pre><code> #include <stdio.h> #include <stdlib.h> #include <math.h> int main(int argc, char* argv[]) { printf("%.0lf\n", pow(pow(pow(pow(pow(2, 2), 2), 2), 2), 2)); return EXIT_SUCCESS; } % gcc -o main -std=c18 -O2 -pipe -fPIC -fno-plt -fstack-protector-strong main.c && ./main 4294967296 </code></pre> Julia:<pre><code> julia> 2^2^2^2^2 0</code></pre>

评论 #32807305 未加载

评论 #32807194 未加载

评论 #32807682 未加载

toxikover 2 years ago

Python isn’t bad enough to give up. Python is nice.

评论 #32806880 未加载

评论 #32807123 未加载

fulafelover 2 years ago

This is not how language popularity works as a cultural phenomenon. There are lots of technically good languages for various purpouses that aren't used much.

ninjinover 2 years ago

First of all, adding “yet” makes it somewhat of a loaded question. The demise/success of Python or Julia will never be a given until after the fact.An initial disclaimer. I am an academic that has been involved in the Julia community for many years and written plenty of Julia code myself. In addition, my entire team uses Python as their day-to-day language to drive research (I do not code nearly as much as I used to these days) and Python was what I used for artificial intelligence research for many years as a student and researcher.Looking at the other comments, I find that most focus on language features, implementation aspects, ergonomics, etc. I think this is a distraction for machine learning code, at the very least when it comes to research. Rather, I think we need to ask: What determines success for a machine learning researcher/practitioner?Firstly, if we turn back the clock ten or so years, most models were written from scratch. This was the time when there was an active “battle” between Java, Matlab, and Python for mindshare. With the shift to deep learning (and GPGPUs), big libraries/frameworks have taken over as these models can be easily composed from primitives of varying degrees and coupled with training algorithms, regularisers, losses, etc. Pre-trained models have also become increasingly commonplace and although they can be ported between languages, the safest bet is of course to go with the training/inference code that the original authors used: Most likely, Python. Julia is putting up a decent fight here against libraries/frameworks backed by some of the richest corporate entities on the planet, but it makes were little sense for most people to go against the grain here as their success is determined by how fast (think man hours) they can put together experiments or push things into production. Dare I say that no one ever got fired for picking Python?Secondly, if you are a Julian and at this point decry how sluggish the snake is, how its implementations are suboptimal stacks across language boundaries, has ancient ergonomics, etc. You are most likely missing the point. I doubt that this battle is won based on any of these “nice-to-haves”. I had the pleasure to sit down with among others Flux’s creator back in 2016 and what I said then I believe still holds today: “We can not just better express what models others can express, we need to enable the creation of new models which other languages can not easily express”. We have seen Julia gain some traction when it comes to Neural Differential Equations for example, which I believe is akin to what I am talking about here. Perhaps there is hope in Julia having a better interface when it comes to bridging symbolic and non-symbolic models, but ultimately it is difficult to predict the future in terms of which models will succeed and dominate the field.To me, if we focus solely on the language itself Julia is largely superior. It has better ergonomics to express machine learning (well, maths in general), a better packaging system (the fact that I have known Python for more than 15 years and it still struggles when it comes to deployment boggles my mind), etc. But, at the end of the day, I think that the delta here is simply not big enough – yet? – to motivate the great majority of researchers and practitioners to make the jump and I respect what I believe is a very rational choice on their part.

Galanweover 2 years ago

As someone working on machine learning, data mining, etc:I would prefer as much as possible that we have the leanest tech stack possible. That means reducing the number of languages, libraries, etc to a sane amount.Is this view, Python is a very good candidate for data science.You can keep the same language for your data science logic and scientists, as well as for your more down to earth developments on infrastructure/techie stuff.

cfcf14over 2 years ago

- Unreliable and buggy core libraries (basic fp math, stats, autodiff) which seems to stem from an academic-style disinterest in focusing on the boring bits of language foundations- 1 indexed arrays- extremely slow start up times (1 min??)- no way to create portable static binary apps (disadvantage vs rust/go)- lack of tutorials, demos- julia is not really faster for math when compared to jax/tf/pytorch/etc anyways

revskillover 2 years ago

Because researchers don't need to spend much time on software engineering best practices. Just script, test and done.

BooneJSover 2 years ago

I was in HPC for a long time - Fortran was and is still popular! It’s a virtuous cycle. Science students learn Fortean because it’s what their professors know. HPC has a lot of regular scientists, not computer scientists, doing their own research at huge scale.Is it similar in ML?

bjourneover 2 years ago

> Unlike Python, it is less headachy. Python has weird behaviours scattered all over the place.<a href="https://www.youtube.com/watch?v=1vBesOFURek" rel="nofollow">https://www.youtube.com/watch?v=1vBesOFURek</a>

xpeover 2 years ago

Most people tend to be risk averse. Relatedly, for _B_ to beat _A_, being somewhat better is not good enough.

fsiefkenover 2 years ago

Because there are languages and libraries that are a better fit for some use cases like for example PyTorch or JAX. The latter has been used in for example the research below.A Differentiable Neural-Network Force Field for Ionic Liquids. (J. Chem. Inf. Model. 2022)H. Montes-Campos, J. Carrete, S. Bichelmaier, L. M. Varela, and G. K. H. MadsenCorrelation Tracking: Using simulations to interpolate highly correlated particle tracks. (Phys. Rev. E. 2022)E. M. King, Z. Wang, D. A. Weitz, F. Spaepen, and M. P. BrennerOptimal Control of Nonequilibrium Systems Through Automatic Differentiation. M. C. Engel, J. A. Smith, and M. P. BrennerGraph Neural Networks Accelerated Molecular Dynamics. (J. Chem. Phys. 2022) Z. Li, K. Meidani, P. Yadav, and A. B. FarimaniGradients are Not All You Need.L. Metz, C. D. Freeman, S. S. Schoenholz, and T. KachmanLagrangian Neural Network with Differential Symmetries and Relational Inductive Bias.R. Bhattoo, S. Ranu, and N. M. A. KrishnanEfficient and Modular Implicit Differentiation.M. Blondel, Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares-López, F. Pedregosa, and J.-P. VertLearning neural network potentials from experimental data via Differentiable Trajectory Reweighting. (Nature Communications 2021)S. Thaler and J. ZavadlavLearn2Hop: Learned Optimization on Rough Landscapes. (ICML 2021)A. Merchant, L. Metz, S. S. Schoenholz, and E. D. CubukDesigning self-assembling kinetics with differentiable statistical physics models. (PNAS 2021)C. P. Goodrich, E. M. King, S. S. Schoenholz, E. D. Cubuk, and M. P. BrennerSome threads discussing the merits of JAX, PyTorch and Julia:<a href="https://news.ycombinator.com/item?id=30349687" rel="nofollow">https://news.ycombinator.com/item?id=30349687</a><a href="https://news.ycombinator.com/item?id=29682507" rel="nofollow">https://news.ycombinator.com/item?id=29682507</a> <a href="https://www.reddit.com/r/MachineLearning/comments/shsfkm/d_current_state_of_jax_vs_pytorch/" rel="nofollow">https://www.reddit.com/r/MachineLearning/comments/shsfkm/d_c...</a> <a href="https://www.reddit.com/r/MachineLearning/comments/st8b11/d_should_we_be_using_jax_in_2022/" rel="nofollow">https://www.reddit.com/r/MachineLearning/comments/st8b11/d_s...</a><a href="https://www.assemblyai.com/blog/why-you-should-or-shouldnt-be-using-jax-in-2022/" rel="nofollow">https://www.assemblyai.com/blog/why-you-should-or-shouldnt-b...</a><a href="https://archive.ph/FL6hs" rel="nofollow">https://archive.ph/FL6hs</a>

dev_0over 2 years ago

Julia is used by exotic firms like Jane street

评论 #32807177 未加载