Only a few months ago people saying that the deep learning library ecosystem was starting to stabilize. I never saw that as the case. The latest frontier for deep learning libraries is ensuring efficient support for dynamic computation graphs.<p>Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we're processing text, one example being a few words while another being paragraphs of text, or when we are performing operations against a tree structure of variable size. This problem is particularly prominent in particular subfields, such as natural language processing, where I spend most of my time.<p>PyTorch tackles this very well, as do Chainer[1] and DyNet[2]. Indeed, PyTorch construction was directly informed from Chainer[3], though re-architected and designed to be even faster still. I have seen all of these receive renewed interest in recent months, particularly amongst many researchers performing cutting edge research in the domain. When you're working with new architectures, you want the most flexibility possible, and these frameworks allow for that.<p>As a counterpoint, TensorFlow does not handle these dynamic graph cases well at all. There are some primitive dynamic constructs but they're not flexible and usually quite limiting. In the near future there are plans to allow TensorFlow to become more dynamic, but adding it in after the fact is going to be a challenge, especially to do efficiently.<p>Disclosure: My team at Salesforce Research use Chainer extensively and my colleague James Bradbury was a contributor to PyTorch whilst it was in stealth mode. We're planning to transition from Chainer to PyTorch for future work.<p>[1]: <a href="http://chainer.org/" rel="nofollow">http://chainer.org/</a><p>[2]: <a href="https://github.com/clab/dynet" rel="nofollow">https://github.com/clab/dynet</a><p>[3]: <a href="https://twitter.com/jekbradbury/status/821786330459836416" rel="nofollow">https://twitter.com/jekbradbury/status/821786330459836416</a>
It's a community-driven project, a Python take of Torch <a href="http://torch.ch/" rel="nofollow">http://torch.ch/</a>. Several folks involved in development and use so far (a non-exhaustive list):<p>* Facebook
* Twitter
* NVIDIA
* SalesForce
* ParisTech
* CMU
* Digital Reasoning
* INRIA
* ENS<p>The maintainers work at Facebook AI Research
At this point I've used PyTorch, Tensorflow and Theano. Which one do people prefer? I haven't done a ton of benchmarking, but I'm not seeing huge differences in speed (mostly executing on the GPU).
I've never fiddled with machine learning thing so don't know anything about it.<p>I am wondering if CUDA is mandatory for torch installation ? I use a Macbook air which doesn't have graphics card, so not sure if torch can be installed and used on my machine.
It is worth adding that there is a wip branch focused on making PyTorch tensors distributable across machines in a master-workers model: <a href="https://github.com/apaszke/pytorch-dist/" rel="nofollow">https://github.com/apaszke/pytorch-dist/</a>
Guess there's no escaping Python. I had hoped Lua(jit) might emerge as a scientific programming alternative but with Torch now throwing its hat into the Python ring I sense a monoculture in the making. Bit of a shame really because Lua is a nice language and was an interesting alternative.
Every time I decide I'm going to get into Python frameworks again, and I start looking at code, and I see people making everything object-oriented, I bail<p>Just a personal (anti-)preference I guess