More often than not, I start my projects thinking that they would be small and don't need a comprehensive framework like Django. Then I figure I need an authentication mechanism, SQL abstraction, an admin site, a pipeline to handle, verify the requests and so on. Then I switch back to Django mostly because I don't want to rediscover the world again. Surely given enough time I could rediscover, but why would one bother?<p>As per staying <i>lean</i> as stated in the article. I don't think that's staying lean at all. I must mention that I have no knowledge of the framework or codebase of the company in question, although rewriting a codebase is almost never a good idea. Well, it will require a lot of rewriting which is redundant, you would be ditching a lot of testing and bug fixing you have done ever since, and you will probably create a space a lot of new bugs.<p>I believe a better approach would be componentize your structure in a way that's still governed by django, every component split into apps. And if there are outlier apps that do not need to be within Django at all could be used by redirecting the requests to those services on the load balancer / reverse proxy level.
This makes no sense from a business perspective. Letting developers work using any languages and technologies sounds like a good idea to individual developers maybe but how do you intend to maintain it? If you stick to a few core languages and technologies and hire / train for those tech then you reduce your risk if one or more developers leave and reduce the total complexity of your stack. You should have a good reason for introducing a new language or technology to your stack and think carefully before diving in. Just create separate Django apps and only switch to a different tech if Django is particularly ill suited for the task.
I don't get it: why does moving to microservices mean ditching Django? Couldn't you take your Django monolith and break it into separate (still-Django) microservices? I get that Django is seen as "too heavyweight" to do such a thing, though there are literally books (<a href="http://shop.oreilly.com/product/0636920032502.do" rel="nofollow">http://shop.oreilly.com/product/0636920032502.do</a>) written about how to slim down Django.<p>I get that monolith to microservice is all the rage these days, I just don't understand (and the article doesn't really say) why you have to "ditch Django" to do that.
I think the biggest reason people are switching to microservices is that it enforces protocols and documentation between components that would otherwise never be created.<p>Microservices create a boundary and that forces you to create a request/response protocol to enforce that boundary. You also have to create documentation for it to be useful.<p>There is nothing stopping you from creating your app as a bunch of small services without the added complexity of microservices/SOA. Each slice of your system could be an isolated library with a stronger enforced protocol and documentation and you'd get largely the same benefits without having to deal with the deployment headache.<p>The reason people don't do that is if they build isolated components in the same monolitic system, nobody takes the time to create a proper protocol and documentation.<p>The only way people seem to be willing to give separate services the appropriate design respect is to physically separate them and solve them as an isolated problem. The physical separation forces better behavior for microservices to even be useful.<p>It doesn't help that most languages have really weak capabilities to enforce strong protocols/boundaries beyond a simple type checker via complier. A language like JS, Python, PHP, Ruby, etc. don't even have a compiler so the ability/desire to create strong protocols in most code is almost nonexistent to most developers.
> A microservices architecture is (fairly) new and it doesn't tie you to a particular language. If you want to build a service in Lisp, Go, Elixir, etc then you can (assuming it makes sense). Good devs love to learn, experiment and hate being bored. A microservices approach encourages all of that<p>The developer in me kind of likes that - fun new toys! But from a business point of view, I'm not sure it sounds like such a good idea: lots of weird, unmaintainable stuff.
Judging by the following statements: "we don't worry much about code quality", and "lots of moving parts isn't without its complexity", I'm just going to very politely wish him luck in his endeavors, and maybe make some popcorn.
One of my previous employers had let each new incoming VP of engineering use whatever he wanted to build out new parts of the app. With the high turnover rate they had, the app turned into the worst Frankenapp piece of crap imaginable. Significant portions of the app were built in php, java, python, javascript, perl, ruby, ruby on rails, mysql, sybase, and a few more I'm forgetting.<p>The result was that changing the title on the homepage took two weeks of manual regression testing and it took a new developer a minimum of two weeks to get a development instance up and running. I felt so sorry for the Ops guys, I can't even imagine how terrible it was for them. There was no one, ops or dev, that knew even 25% of how the whole system worked. It always felt like the leaning tower of pisa made toothpicks and popsicle sticks held together with bubble gum and boogers.<p>The cost to run that Rube Goldberg machine monstrosity ruined what could have been an amazingly profitable business and great place to work. I don't recommend it.
> it was becoming incrementally larger and more complicated for new team members. Because of this, the monolithic Django system was already proving a headache to get quickly set up, running and understood by our new hires.<p>So, now you have thirty smaller services for each new hire to install and link together? I can all but guarantee that this particular problem is better solved with Vagrant and some orchestration scripts. Takes your install from a list of packages to a single "vagrant up" command.
I've been a big fan of microservices for years. I believe that a good microservice has a lot in common with traditional Unix commands. And I find it funny that no one here has gone off on a tangent to talk about how horrible those are.<p>Unix's tools and good microservices have the following things in common:<p>1. They are small
2. They are loosely coupled
3. Shared-nothing state
4. They enforce standards for consuming and emitting data
5. They enforce a single standard for piping data from one process to another.<p>That said, I strongly believe that this won't work for every class of problem. And while I may be putting words into Tom Watson's mouth here, I believe he would say the same. But hey, guess what? This WOULD work for quite a few problems!<p>Some of the sentiment in the comments here feel defensive and knee-jerk. And I don't understand why. Tom Watson isn't telling you that you should use microservices. He's only explaining why he's using it at Hubble. I'm pretty sure how it turns out is going to have no impact on anyone else here, except (obviously) his customers.<p>But I hope it succeeds. I think there's a lot we can learn by applying the lessons Unix taught us to web-scale utilities.
Doesn't Django already encourage this type of design? Each Django project is composed of several installed 'apps'. Each app should be portable enough that it could be the only app in any project.
> We don't worry too much about code quality as I believe that that is fixed at the recruitment level. Good people write good code and, more importantly, ship good products.<p>Good people make mistakes too. Having good tools to ensure code quality doesn't make it impossible to ship bad products, but it's definitely important.
What about the headache of monitoring 10s of microservices, making sure they stay up and running and healthy?<p>I assume that REST or Thrift (or something similar) is the way most microservices talk to each other. What if the latency isn't acceptable?<p>I've thought about switching our architecture to microservices but these two potential issues are making me hesitate.<p>How important are these issues in practice?
It's amazing how easily people swap "I can't see how to use this in the contexts I'm familiar with" to "it doesn't make any sense at all".<p>Being myself - just a few years ago - a long time strong proponent of tightly coupled monoliths, I can't believe how blind and foolish I was in my fundamentalist rejection of loosely coupled architectures.<p>Monoliths do just fine up to a certain level of structural complexity. Above that, asynchronous service-like architectures are the only viable way to go.<p>Microservices is an attempt to see if the services patterns work below that waterline, down to the function level. That's why I find microservices at least interesting.<p>At any rate, it's a cost/benefit balance game, not an ideology.<p>Edit: clarity.
We had similar issue's.
Our Designers couldn't design with that big Django app, so we split it up.
Everything talks ReST to a cached AngularJS Client.
We still have Authentication in Django and two of our Microservices are running with Django aswell. Some things were reimplemented in Golang, as we tried some new things out.<p>What people here are talking is mostly dumb, since Microservices are better in a lot of ways especially when it comes to multiple Developers.
Currently my company struggled a lot with Code that got changed by two guys at the same time, which was hard to fix.<p>We had less unit tests than now, since they were too complicated at a certain time.
Standardization is, generally, good for maintenance, learning curve and security, not always for retention (fun) though. Too much standardization is like the flip-side of too many one-offs: an extreme that can lead to unintended consequences. It's much easier to accrue technical debt and organizational bifurcation by having too many languages, with a big, painful shake out (technical debt repayment) into just a 2-3 (say Ruby|Python|Node & Go|Scala|Haskell|Erlang|C)... (I've seen this happen at a number of shops go through growing pains at scale).<p>Overall though, services should expose either a well/self-documented RESTful, zmq or similar API such that they can be rewritten in (your favorite whatever here). Being able to discover the API meta information automagically can be helpful for autogenerating clients.<p>The other aspect is that services have a risk budget: if the service is conventional and not as crucial, increase the experimentation. If the business/process model is risky/uncertain, use common, stable technologies.<p>(Gist: Pick from the toolbox, carefully... apply common-sense... Don't get stuck on perfect now or perfect waay later... Anticipate a little now to avoid pain of changing later.)<p>PS: <i>Enterprise Rails</i> is a great read. It's safe to ignore the Rails bits, because the architecture bits are universal and excellent.<p>Update: RESTful API metadata formats <a href="http://apiux.com/2013/04/09/rest-metadata-formats/" rel="nofollow">http://apiux.com/2013/04/09/rest-metadata-formats/</a> (and also <a href="https://en.wikipedia.org/wiki/HATEOAS" rel="nofollow">https://en.wikipedia.org/wiki/HATEOAS</a> )
Every time I see microservices, I think of Martin Fowler's First Law of Distributed Object Design: "Don't distribute your objects." <a href="http://martinfowler.com/bliki/FirstLaw.html" rel="nofollow">http://martinfowler.com/bliki/FirstLaw.html</a><p>Splitting your app into multiple apps that send each other HTTP requests or MQ messages seems likely to just make your app slower.
Can I be the first to say that this is a terrible idea?<p>>We have a multitude of parts that each could easily be self contained as their own service. For example: Messaging, Search, Authentication, etc. These all have different requirements and focuses and therefore lend themselves better to different languages.<p>Yeah, not really. Python is not inherently less suitable for authentication than, say, Java. Both languages are designed for general purpose programming. The only real drawback to python is that it is relatively slow and this isn't even likely to be an issue at all if you are using django (you'll most likely bottleneck on I/O).<p>>Also, looking at our roadmap, we have lots of sprints and projects coming up that could be implemented in their own silos. By splitting these into services this allows us to quickly test and iterate without worrying about the rest of our product.<p>And splitting them into separate libraries is not sufficient because?<p>>Whereas in a monolithic system these services are bundled together and can create a spaghetti mess of workarounds and packages.<p>Newsflash: you're MORE likely to make a spaghetti mess of workarounds if you use a microservices architecture because of the impedance mismatch between API layers. You'll have to create a representation of, say, a user object on both sides of the microservice divide and write code to serialize it and deserialize it. Hello massive code repetition and obscure bugs.<p>>Having been a dev team of 1 for a long time I'd never experienced the headache of installing all the dependencies required by our Django system. It never really occurred to me that as I installed more packages, it was becoming incrementally larger and more complicated for new team members.<p>If you think your headache is bad with django (which has consistent opinions - e.g. on how templates should work - that makes it possible for modules to work together), get ready for a nightmare when you start using a hodgepodge of different systems written in different languages.<p>>We don't worry too much about code quality<p>This has officially moved this blog post from bad idea to outright parody.<p>>However, most of the cons are part and parcel with programming.<p>Here's a small laundry list of the exciting issues you'll now get to deal with:<p>* Writing serializers and deserializers for your objects so they can be passed from one service to another. And debugging them!<p>* Babysitting all the new services you will need. You'll need to keep them up and running, monitor them, have a backup plan for all of them going down and lots of lovely extra error conditions.<p>* The headache of trying to figure out how to scale your network of microservices - probably meaning lots of time configuring load balancers.<p>And much, much more.
make sure to document well. When you're scaling up a team and moving from a coupled code base to independent codebases a good amount of assumption happens, especially around how mature each codebase is.
> For example: Messaging, Search, Authentication, etc. These all have different requirements and focuses and therefore lend themselves better to different languages.<p>What?<p>Every single one of those is a solved problem that has bindings to most major languages. Why in the world would you switch languages for an Auth mechanism (for example)?
> Having been a dev team of 1 for a long time I'd never experienced the headache of installing all the dependencies required by our Django system. It never really occurred to me that as I installed more packages, it was becoming incrementally larger and more complicated for new team members. Because of this, the monolithic Django system was already proving a headache to get quickly set up, running and understood by our new hires. Splitting it up into more digestible chunks (services) therefore should allow us to scale the team a lot quicker this year and beyond.<p>Yet, the dynamic language community constantly laughs at Java developers and their high overhead dependency management solutions, which just happen to have these kinds of problems solved since more than a decade.<p>Pathetic.