Hot reloading is next to impossible to do safely in the general case in Python, and this library is really basic and doesn’t really do anything special.<p>The only case where it does work is if you maintain a single reference to the module you wish to reload, _and_ the target module doesn’t load any C extensions, _and_ it doesn’t create state outside its module (very easy to do accidentally).<p>That’s why every autoreloader implementation restarts the whole process when any change is made, because how would you even detect that it was safe to hot reload something in the general case let alone actually hot reload it safely?<p>If you need to add an autoreloader to your project use Hupper[1]. If you’re weird like me and you’re interested in autoreloaders, check out the Django[2] and Flask[3] implementations or my talk[4] for more details.<p>1. <a href="https://pypi.org/project/hupper/" rel="nofollow">https://pypi.org/project/hupper/</a><p>2. <a href="https://github.com/django/django/blob/master/django/utils/autoreload.py" rel="nofollow">https://github.com/django/django/blob/master/django/utils/au...</a><p>3. <a href="https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py" rel="nofollow">https://github.com/pallets/werkzeug/blob/master/src/werkzeug...</a><p>4. <a href="https://youtu.be/IghyoR6ld60" rel="nofollow">https://youtu.be/IghyoR6ld60</a>
Because of the nature of Python, using importlib.reload() is going to lead to terrible bugs at some point. Use importlib.reload() only in the shell, for convenience.<p>If you want autoreload for dev (e.g: like django dev server), better restart the entire process.<p>There is a 7 years old project that can help you with that called watchdog:<p><pre><code> https://pythonhosted.org/watchdog/
</code></pre>
It's a library that uses the fastest way available (e.g: inotify,kqueue...) to react to file changes (you can filter by types such as create, delete, write, etc) and triggers whatever code you want, passing you the metadata.<p>If you just want something very simple, you don't have to code, you can install watchdog with its optional watchmedo dependancy:<p><pre><code> pip install watchdog[watchmedo]
</code></pre>
It will give you the watchmedo command, that lets you do something like this:<p><pre><code> watchmedo shell-command --patterns="*.py" --recursive \
--command='python your_script_to_rerun.py' .</code></pre>
Does this "just work"? How does it handle classes?<p>I've been writing my python code in a style that can be hotswapped. Roughly, I use global functions and pass around structs, like C. The function calls are made with `api.foo(data)` instead of `data.foo()`. Then I can just replace `api.foo`'s definition with whatever I want after attaching with a debugger. Example: <a href="https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d6cfdc3192aba0f2b4/train_multi.py#L475-L499" rel="nofollow">https://github.com/shawwn/gpt-2/blob/e4868225382e8a475ef3c3d...</a><p>I assume there's a much better way to do this, but the class system seems to make it hard to swap things out. Doesn't "isinstance" break if you redefine the class?<p>EDIT: Oh, this isn't actual hot-reloading for python. Darn. I'm really hoping someone will tackle that... Cool project though!<p>EDIT2: Actually, this project <i>does</i> seem to reload the modules: <a href="https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7ccdb546fcef4553617f9/hotreload/reloader.py#L47-L48" rel="nofollow">https://github.com/say4n/hotreload/blob/0cf3a0b466466f99f8c7...</a> ... so the original question stands.<p>By the way, you may want to use `import traceback; traceback.print_exc()` to log exceptions. I'm not sure how that plays with python's `logger` class though.
For comparison, here’s Werkzeug’s reloader implementation: <a href="https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_reloader.py" rel="nofollow">https://github.com/pallets/werkzeug/blob/master/src/werkzeug...</a> (when you flask run or call app.run with reloader turned on, it’s handled by this module.)
As other have pointed out, the best way is to restart the process. If you are looking for hot reloading you should look at programming languages that run in BEAM like Erlang and Elixir[1].<p>There are some gotchas with hot swapping, but it's designed and used in production for many years [2].<p>If somebody knows other languages/systems that do let me know.<p>1. <a href="https://elixir-lang.org/" rel="nofollow">https://elixir-lang.org/</a>
2. <a href="https://stackoverflow.com/questions/37368376/how-does-erlang-hot-code-swapping-work-in-the-middle-of-activity" rel="nofollow">https://stackoverflow.com/questions/37368376/how-does-erlang...</a>
You can do something like this on Linux:<p><pre><code> inotifywait -m myfile.py -e 'CLOSE_WRITE' |
while read line; do python3 myfile.py; done
</code></pre>
It will be more responsive than a busy while-sleep loop.
How is this different from using Python Watchdog?<p><a href="https://pythonhosted.org/watchdog/" rel="nofollow">https://pythonhosted.org/watchdog/</a>
Looks like it uses hashlib to check the file for changes, which is a nice way to make it cross-platform compatible.<p>For those who run on linux, you can use the inotify tools to set up a watch on a file (or directory or recursive directories). inotify uses kernel events and won't require re-reading and re-hashing the file.
Don’t hot swap. Just don’t do it. 99.99% of us should never need to do it. Learn sound network topology and load balancer practices instead. Embedded? Your hardware documentation will tell you how to offload this problem to it. There’s no earthly reason for most of us to be learning this.
having an infinite loop reading the file and calculating its sha checksum every 1s is not ideal. I'd suggest using the OS premitives to get notifications when the file is touched (inotify on linux and ReadDirectoryChangesW on windows), or simply use the watchdog lib
When I add autoreload functionality to my programs I just spawn a thread that checks for last modified date and then re-executes the program with os.execlp(sys.argv[0], *sys.argv)<p>I found my solution superior in several ways:<p>- I can force an autoreload just by saving a file (no file changes needed to force an md5 diff)<p>- exec() doesn't suffer from the side effects of not restarting the python interpreter, clean start every time :)<p>- it is also quite portable and doesn't require extra dependencies like inotify/fswatch/etc.
</ul>
Sorry, but that's not really hot-reloading. It's only reloading a file.<p>Why do we only get hot-code-replacement and edit-code-and-continue-debugging from microsoft and sun(oracle)?
Not keen on hashing the file every second but I understand why... I'd rather it used filesystem modification events and responded accordingly. But that might more more complex than what you intend.<p>But for simple scripts being live-coded that need to be quickly "up" and hot-swapped on rapid modification this looks fine. I could always increase the delay to a minute or hour or more if the changes were only occasionally changed.
By the way, we had this discussion about Erlang's hot reloading five years ago.
<a href="https://news.ycombinator.com/item?id=10669131" rel="nofollow">https://news.ycombinator.com/item?id=10669131</a><p>Erlang runs on a VM designed for hot reloading so the comparison is somewhat unfair to Python which was not designed for that.
For those using Bazel, there's ibazel:<p><a href="https://github.com/bazelbuild/bazel-watcher" rel="nofollow">https://github.com/bazelbuild/bazel-watcher</a><p>It's great, I use it all the time, especially with unit tests.