Related to this, if you set the env var `PYTHONPROFILEIMPORTTIME=1` or run python with `-X importtime`, it will print out the cumulative and self times to import various modules.<p>There is then this neat tool to visualize the data. <a href="https://kmichel.github.io/python-importtime-graph/" rel="nofollow">https://kmichel.github.io/python-importtime-graph/</a><p>Highly recommend to find the worst imports affecting your program startup time.<p>In general, the python community values tend towards functionality over performance. For example, large modules (looking at networkx here) will often import a bunch of there submodules in their __init__.py, which means all modules now end up loaded even if you didn't need them.<p>I've never tried <a href="https://pyoxidizer.readthedocs.io/en/stable/oxidized_importer.html" rel="nofollow">https://pyoxidizer.readthedocs.io/en/stable/oxidized_importe...</a>, but it compiles all the imports into one, memory mapped file, that _may_ speed up the importing.<p>Having everything compiled to bytecode also helps a bunch.
I've been writing python for going on 20 years now and while it was a good language to cut my teeth on thus sort of analysis brings only horror. Many thanks to the author for dropping into plain view.<p>I'm going to go back to learning more C and Forth... And shake my fist at passing clouds :)
I also wrote a somewhat similar tool. I call it deep-ast. It's pretty flexible in what it can track. I used it when refactoring some code in urllib3, to see what Exceptions could get raised along a given code path.<p><a href="https://github.com/DontShaveTheYak/deep-ast">https://github.com/DontShaveTheYak/deep-ast</a>
Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue.<p>I look at posts like this and cry.
If it's a large project, I'll use local imports to defer this cost only around where I'm plotting. That way, if I have another entry point that only does computation or is part of a larger system like a web application, it won't have this sort of overhead.