You Don't Need to Rebuild Your Development Docker Image on Every Code Change

I have to scratch my head every time I see a blog post about creating a dev environment that doesn&#x27;t use volumes like this. IMO, this is the way to go and what Docker was meant to be as a dev environment.<p>Maybe it&#x27;s because I&#x27;m never confident in the changes I make, but my style is to write a few lines at a time, save, re-run or refresh. I can&#x27;t imagine having to do a docker build every code change. If you do the later, it&#x27;s just nominally better than developing on a remote server, which brings its own challenges.

Docker truthfully told is unusable for Dev envs that need to change constantly. It was never built to handle this use case. I haven&#x27;t seen a good docker compose file that can be reliably used as a development environment in multiple oses with good performance. There&#x27;s all sorts of edge cases where volumes don&#x27;t work. In my opinion docker is very useful for thing like dbs, queues and other processes where the underlying code doesn&#x27;t change. But for everyday frontends and backends it&#x27;s not worth it. Nowadays I write a shell.nix file which contains all the dependencies the project needs, it works but is definitely not as easy to learn as a docker file

Hi HN! First time I see one of my articles on here. What a nice surprise.<p>If the above link caught your attention, you might also enjoy the following ones:<p>* For the quickest ROI: <a href="https:&#x2F;&#x2F;vsupalov.com&#x2F;improve-your-docker-images&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vsupalov.com&#x2F;improve-your-docker-images&#x2F;</a><p>* Stuff I WISH I knew: <a href="https:&#x2F;&#x2F;vsupalov.com&#x2F;12-docker-facts&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vsupalov.com&#x2F;12-docker-facts&#x2F;</a><p>* If your image builds are slow: <a href="https:&#x2F;&#x2F;vsupalov.com&#x2F;5-tips-to-speed-up-docker-build&#x2F;" rel="nofollow">https:&#x2F;&#x2F;vsupalov.com&#x2F;5-tips-to-speed-up-docker-build&#x2F;</a><p>Looking forward to join the discussion later!

I have done this but if possible I just run my code locally and reserve docker for infra (e.g. DB). Java, nodejs, go are all very easily run locally.<p>Maintaining a complex Dockerfile is just more crap that makes programming tedious.

Works remotely on kubernetes, for us the game changer was running inotify_wait on cygwin for our windows users, changes are up in about 600ms.<p>1. Configure a service 2. Use start command sleep infinity 3. Install inotify_wait on windows 4. Do a loop like this<p>Look for changes, Rsync changes to pod, Pkill java, Run start.sh<p>We do a grep on changed files and only kill java if jar&#x2F;classes and other deps has changed, that makes it possible to edit html in pod and get fast updates.

This is great, until you need to watch a tree of files in macOS. Then docker takes a core to do the fsnotify&#x2F;watch.<p>There are caching settings to make it better, but there are race conditions between the watch and when file contents change, so sometimes the JS stack will compile an inconsistent file.<p>OTOH, on linux, it&#x27;s grand.

While this is great for people with a fundamental understanding of containers and your prod environment this will usually lead to some issues with developers that don&#x27;t need to, or want to, have context in these areas.<p>In the past, to make a very similar workflow possible, I&#x27;ve built tools that automatically watch your source files and rebuild &amp; restart only what is needed [0]. This was built for bazel + docker-compose but there isn&#x27;t a reason one couldn&#x27;t watch the &quot;build:&quot; contexts for what files are important.<p>At a previous company one of our engineers was a huge fan of this volume mount approach and every single time something broke (which was very frequent due to some prod&#x2F;dev env magic we had) I had to assist quite a few more junior devs figure out what was wrong with their machine. For those with scripting languages, was it their system&#x27;s newline endings? For compiled languages, was their system SDK different then what was in the container? For prod bugs, did they forget to rebuild &amp; test the container before opening their PR (we had no automated integration testing)?<p>In my opinion, if you can make your build system in charge of building&#x2F;packaging things you&#x27;ll have a much happier time.<p>[0] - <a href="https:&#x2F;&#x2F;github.com&#x2F;CaperAi&#x2F;bazel_compose" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;CaperAi&#x2F;bazel_compose</a>

For devs on macOS, keep in mind that your filesystem is case insensitive!<p>If you&#x27;re using a linux image that expects a directory like `UpperCase` and you named it `uppercase` locally it would work.. until you bake the source for your production release and you get errors around that directory not existing.<p>&quot;But it works locally!&quot;

<a href="https:&#x2F;&#x2F;code.visualstudio.com&#x2F;docs&#x2F;remote&#x2F;containers" rel="nofollow">https:&#x2F;&#x2F;code.visualstudio.com&#x2F;docs&#x2F;remote&#x2F;containers</a>

A little known feature is that the Docker integration in Visual Studio gives you this automatically.

Nix has saner repeatable dev envs, and makes updating or adding deps a snap.<p>Can be used to build scratch containers. Can be used to cross compile.<p>The only major downside is it requires learning and understanding nix which I get is a hurdle, but one that&#x27;s well worth it.<p>Docker is only <i>one</i> container tool of many now, and it&#x27;s worth exploring what else is out there.

In the past few weeks, I have spent some time and released dew [0]. It helps encapsulating this kind of setups in configuration and minimising typing. dew is still evolving, but it has served me well.<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;efrecon&#x2F;dew" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;efrecon&#x2F;dew</a>

For any rails dev out these<p>volumes:<p><pre><code> - .build&#x2F;.bundle-cache-dir&#x2F;app:&#x2F;usr&#x2F;local&#x2F;bundle&#x2F;</code></pre>

Skaffold is a great tool for this and employs hot reloading techniques without having to rebuild the image. <a href="https:&#x2F;&#x2F;skaffold.dev&#x2F;" rel="nofollow">https:&#x2F;&#x2F;skaffold.dev&#x2F;</a>

We wrote a tool to make using development Docker images easy - <a href="https:&#x2F;&#x2F;metaswitch.github.io&#x2F;floki&#x2F;" rel="nofollow">https:&#x2F;&#x2F;metaswitch.github.io&#x2F;floki&#x2F;</a>

Another thing I&#x27;ve recently found helpful was docker layer caching inside github actions. Pretty easy to integrate, saves a lot of build minutes.

I&#x27;ve been doing this for about 6 months now, it saves a lot of time. Especially if you need to upload changed docker images to AWS.

I have meant to try if VMWare&#x27;s vctl is any better regarding detecting file changes