I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming.<p>On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages. You can't even reliably do a "find usages" of classes using a dynamically typed language.
I pretty much agree with everything in the article - except for the bit where he tries to quantify why python is better from a developer efficiency perspective than other languages.<p>The main example he cites is a study that compares the amount of time writing string processing routines in different languages - which is quite a bit different from the work I do every day. I develop web apps which means I generally work in very large code bases, and spend most of my time modifying existing code rather than writing fresh code from scratch. I have found that statically typed languages (java + typescript) and the fantastic IDE support that comes along with them make it really easy to navigate around the code and refactor things. Also - the compiler tends to catch and prevent a whole class of bugs that you might otherwise only catch at runtime in a dynamically typed language.<p>Of course there are other situations where I prefer to use Ruby as my scripting language of choice - it all comes down to using the right tool for the job at hand. Unfortunately I don't think the author gives enough consideration to the trade-offs between static vs. dynamically typed languages, and I think he would have been better just leaving that section out as it isn't really necessary to prove his point that CPU efficiency isn't important in a lot of applications.<p>Ultimately though I completely agree with his main point: "Optimize for your most expensive resource. That’s YOU, not the computer."
Python is also heavily used in science, where performance really does matter. It's successful because of how highly ergonomic python apis can be built on top of optimised C/C++/Fortran libraries.<p>That said, there is clearly a desire to write 'fast' code in python itself without swapping to C. Cython helps, but to get really fast Cython code you actually have to write with C-semantics (so you are basically writing C with Python syntax).<p>Projects like numba JIT are interesting in that they can optimise domain-specific code (i.e. numerical/array code) that's written in normal python style. It also means jumping through a few hoops (although with the latest version in many cases all you need is a single decorator on your hot function). You can even do GIL-less multithreading in some cases.<p>Overall things are looking promising, with the addition of the frame evaluation API and possible improvements to the python C-api that could make JIT and similar extentions easier.
The author argues from his professional experience as a Python developer that it's fast enough, that you'll spend most time waiting for I/O anyway, that you can just throw more servers at the problem etc.<p>The problem is that his experience as a Python developer doesn't accurately reflect the prevalence of problems where runtime CPU performance actually is an issue. Of course not, because who in their right mind would make an informed decision to solve such a problem in Python? Python has worked for him because it is only useless for a category of problems that he hasn't had the opportunity to solve because he's a Python developer. Outside this professional experience, not everything is a trivially parallel web service that you can just throw more servers at if CPU time exceeds I/O waiting.<p>It all really boils down to what your requirements are, whether you have all the time and memory of a whole server park at your hands, or a fraction of the time available in a smaller embedded system, how timely the delivery of the software has to be and how timely it needs to deliver runtime results once it's up and running. There are times where Python just isn't fast enough, or where getting it fast enough is possible, but more convoluted and tricky than implementing the solution in a more performant language. Developer time may be more expensive than the platform that my solution is for, but that doesn't get around the fact that it eventually will need to run with the available resources.
Unless we are talking like circa 1999 I don't think I have heard a complaint yet that Python is slow. I'm curious who or where the author heard that from (not specifically the people themselves but the domain they are in).<p>What I have heard complaints about Python are (and I don't agree with all these points):<p>* Its not statically typed<p>* The python 2/3 compatibility<p>* It has some design flaws: GIL, variable assigning, mutable variables, lambdas, indentation (I don't agree with all these but this is complaints I have heard).<p>* The plethora of packaging (ie its not unified)<p>I guess one could argue its slow because it can't do concurrency well but that really isn't raw speed.<p>Then the author started comparing string processing of programmer time from a study which... doesn't help the authors point at all.<p>* Python has and will always be fast at string processing and most people know this<p>* The people that complain about python speed are almost certainly not doing string processing<p>* I have serious questions about the study in general (many languages have changed quite a bit since then)
"It doesn't matter than Python is slow, besides we can use compiled libraries to speed it up"<p>"People saying it doesn't matter that Python is slow are deluding themselves and preventing Python from getting faster like JS did"<p>"Python is inherently harder to optimize than JS since it has <very dynamic features>"<p>"Smalltalk/Lisp/etc are also very dynamic yet are much faster"<p>"The slowness of Python is harming the planet by being inefficient and therefore wasting more energy/producing more pollution"<p>Did I miss any arguments? I know certain topics are bound to attract some repetitive discussion, but "Python is slow" has been one of the worst.
Python is my Swiss army knife. I love it because it is a single tool that can aid in almost every project I do. But if I'm doing one specific thing a lot, I want that thing to be done well and done efficiently, so I'll reach for the specific screwdriver I need.<p>Also most of my problems are IO bound so single threaded concurrency is fine.<p>But I represent a very small portion of the global problem space.
The fact that Python is slow isn't its only problem. What I care more about nowadays is wasting my time hunting bugs that could have been avoided by a static type system.
Python's value to me has always been that it's easier to get things done, not it's speed. One time when I was interviewing a candidate for a coding job, the candidate said she loved Python the most "because you can just yell at it and it'll work."<p>It's both the breadth of the standard library and ecosystem, and the simple language design, that make developing things in Python faster for me.<p>Doing problems on Project Euler has been an education for me in how algorithm matters more than speed. Lots and lots of people spend hours writing long C++ codes that are easily beaten by a few lines of Python. It certainly goes the other way too, and the wrong algorithm in Python is even that much slower and more painful than the right algorithm in C++. But when the right algorithm is used and the problem is solved in a few milliseconds, it really doesn't matter which language uses more CPU cycles, all that matters is whether you saw the insight that let you skip 99% of the search space, and how much time you spend writing code.
Somewhat ironically, Python is used a lot for things that would benefit from raw speed (data processing pipelines) and do not benefit at all from dynamic typing (since the kind of property bags / data frame views over data are easily replicated in statically typed languages). But Python's C extension API is quite a bit easier than p.e. Matlab's MEX API (to me at least); can typical Python IDEs compile and relink extension modules without an external build step?<p>> Your bottleneck is most likely not CPU or Python itself.<p>With applications that are dominated by raw data processing, it's very, very easy to be CPU dominated. Hell, I had one quite trivial data converter for logfiles where the "parsing the printf string" part of Java's printf dominated processing and writing a custom formatter halved processing time (while regexes can be compiled, the format string cannot be precompiled and will be interpreted each time); it's one of those things where I would intuitively say "why did this moron write his custom formatter" if I stumbled upon it in a code review. Intuitively, you'd expect this to be a simple case of an IO dominated task (which it is now once the bottleneck has been removed).<p>If it's fire-and-forget batch jobs, you can get away with it, but if the converter is part of a user facing fat client application that runs on a old office laptop, you don't have that luxury.
The article could be titled: "Yes, Python is Slow To Refactor and Maintain, and I Still Don't Care".<p>I never understand why dynamic language enthusiasts primarily focus on new code only. You have to discuss all sides of increased or decreased productivity to make a rational argument.
> Your bottleneck is most likely not CPU or Python itself.<p>I've found that this is often the case. Nearly always disk or network. But it's sometimes surprising how little work you need to do to become CPU-bound. This is the price we pay for such a tremendously dynamic language.<p>Indeed, the article's suggestions of C/Cython/PyPy are good ones to remedy the problem when it occurs.
I get the point this guy is making, but if you need something parallel for a cpu bound task, throwing more hardware at the problem isn't the most efficient solution if you can just use more cores. For example adding another quad core when the first cpu is only using one core anyway is inefficient and expensive.<p>Right tool for the right job I suppose.
"It used to be the case that programs took a really long time to run. CPU’s were expensive, memory was expensive. Running time of a program used to be an important metric."<p>As hardware gets faster we give it new tasks that could not be achieved before. Like rendering high resolution stereoscopic images using physically based shading at 90 FPS on relatively cheap consumer hadware (VR). There are still quite a lot of code that we call 'performance critical'. Most of that code is written in C/C++ (and CUDA and glsl, and hlsl, etc...) today.
The problem is not so much that python is slow. It's that in some scenarios python can't be made fast.<p>Fast prototyping is great but being stuck with a prototype for deployment isn't.
There's are still some big gains python could make, if python implementations were better.<p>Micropython is equivalent to a real-time cooperative-multitasking OS. If it had ~~better~~ support for things like cffi, you could implement posix on top of it. I can imagine a laptop that runs gnu+python in the next few years.<p>That's a whole new usecase, simply because that implementation uses a lot less ram. What usecases would we discover for a faster python?<p>Shared objects and proper sandboxing would also be huge.
This keeps getting posted, and while it makes some valid points, it's a lot of handwaving.<p>Arguably, other languages can get code out faster depending on the dev, language, etc.
Many times when Python is blamed for being slow, it's the programmers fault. Python is great that you can 'regular' people writing code in it quickly. The problem is, these regular people don't always understand algorithms or things like caches, threads, databases...<p>A lot of these users can just say "My department needs a $40,000 24 CPU server with maximum RAM from MicroWay/SuperMicro, we need to run our codes faster", when they are just trying to brute force things.<p>They understand the problem domain but don't have the programming skills to use a computer to efficiently solve it.<p>But, these guys are all a step ahead of the ones who are stuck in the mindset of "C is the only language fast enough for my work", while not even understanding pointers and basic syntax and getting stuck on silly things like text processing, which could be done in minutes in Python.
Yes, time to market is important. However, you don't need to compromise convenience of development for the sake of performance. If you twist your Python code to get performance it takes time. If you need performance, and like the syntax of Python then you should take a look at Nim [1]. With Nim I develop as quickly as in Python while I get the performance of C.<p>[1] <a href="https://nim-lang.org" rel="nofollow">https://nim-lang.org</a><p>I believe application performance <i>is</i> important on servers. It makes a difference if your Shop software written in Python is able to handle 50 requests per second, or if the same software written in Nim can handle 500 rps. And by the way, Nim provides static typing which helps a lot to catch errors at compile time.
Premise: It's more important to be productive than to have fast code.
Conclusion: Use Python.
Is the premise true? For many cases- yes, but it depends. If you are running an application on the cloud and your metric is $/user/year and you have many users then saving some compute resources for each user gets attractive and you don't want to just throw another VM at it.<p>Is the conclusion true? Garbage collection gives big productivity gains. Other languages have GC. It's not nice to see your Python code die after a few days because you messed up the type passed to a function. Other languages fix that at compile time. Multicore is now. Other languages are built with better multicore awareness.
> without getting stuck in the weeds of the small things such as whether you should use a vector or an array<p>Yes, instead get into the weeds of tuple vs list<p>Not included in the graph of time-to-solve-problem static languages: statically typed languages with type inference
> However, this is no longer true, as silicon is now cheap. Like really cheap. Run time is no longer your most expensive resource.<p>Our client won't spend more money than a t2.medium instance on aws. Nothing we can do about it. In that case, run time does become an expensive resource.<p>But I get the point that OP is trying to make. Just wanted to mention that not all of us have the comfort of having enough resources on which our app runs.
> It’s more important to get stuff done than to make it go fast.<p>This is not a real absolute. It is only valid when what you have to run will not benefit a lot from performance or suffer a lot from lack of it.<p>The real guidance you can have in these matters is: how many times is my code going to run per second?<p>Some programs are written to be run once a day, others 10000000 times in a second. The first ones should be written in the language you're most productive in, the second ones in the fastest possible language.
Putting aside the discussion regarding productivity, there is a case where I have found execution time to matter. Scaling an application which uses an unsharded database. The long transaction durations and number of connctions were bottlenecking db throughput. The particular app was a Ruby/Rails monolith.
This sentiment is the reason for almost all software (especially on the web) beeing a load of crap.
It's slow, it's buggy and developers always give the same excuse: CPUs and memory are cheap, therefore we can waste our customers time.<p>Imagine what we could do with the amazing hardware we have, if people started to do the sane thing and actually use the hardware to do things efficiently.
Giving EvE Online as an example is bad because that game artificially slow the game loop to keep up with the number of players, would this happen with C++ on a recent architecture? Probably not.
In "What if CPU time is an issue?" we could also mention the nim language (and not only cython) because it compiles (not only) to C and feels like python.