VM Sprawl – Docker to the Rescue!

By | June 17, 2014

Right then. A week or so ago was DockerCon where version 1 of Docker was announced. And personally, I think this is a technology that’s gonna absolutely rock the world!

docker logo

To set the scene, let’s take a quick look at the world we currently live in.

The VM World

There’s no doubt that we live in a virtual world. But more than just a virtual world, I’d say it’s a virtual machine (VM) world! We all know and love VM’s. They’re a damn sight better than the old physical world we used to live in. It’s just that… well…… they’re still a bit clunky!

In fact, the entire VM world is polluted with too much duplication and waste….

pollution

We take a physical server (CPU, RAM and storage etc.), and lash an operating system on top of it. Fair enough, we need to do that. But then, we lash more and more operating systems (Virtual Machines) on top of that. Why!

osontopofosontopofos

Well…. Operating Systems obviously provide abstracted resources for applications, and applications tend not to run on bare metal.

Operating Systems also provide memory management systems, scheduling, and constructs like processes and threads… But the Hypervisor is an operating system, and can provide all of that. So why not leverage those functions directly from the hypervisor (operating system) and run apps directly on there?

The VM World Tax

Well…. that brings us to the other major reason we pile the operating systems on. Basically, most operating systems aren’t so good at running multiple apps concurrently. No issue, we’ve got a simple solution to that – for every app, we install a new OS!

Need 30 apps? That’ll be 30 operating systems to go with them! Ouch!!

And that’s where we meet todays major IT issue – VM Sprawl!

car-sprawl

Yes, managing 30 VM’s is better than managing 30 physical machines. But it’s still easy to see how the model is fundamentally crap – you get a 100% OS tax every time you deploy a new app!

Every VM has an OS, and most of those OS’s need anti-virus, licensing, patching, booting, managing..Argghhhh!!

It’s just not great!

And please don’t give me “well if you deploy VM’s on a deduplicating shared storage array you get huge capacity savings”. That’s barely even papering over the cracks!

If we could install applications without operating systems, surely we would. Or put another way…. if we could safely and securely run 30 applications on a single operating system, surely we would!

Enter Docker

The guys at Docker (formerly dotCloud) realised this shocking pollution in the VM world. And they looked at Linux and thought, there’s a better way to do this! And they leveraged a bunch of Linux kernel features to build Docker.

Essentially, those kernel features were:

cgroups (Control Groups): allow fine-grained control of resource allocation to groups of processes
Namepsapces: provide workspace isolation for PID, IPC, mounts, networking etc. Taking the PID example, namespaces can create isloated and unique views of the PID namespace
Union filesystems: allow multiple separate filesystems to be layersed on top of each other and create a single unified view
Linux Containers (LXC): Basically combines namespaces and cgroups to create isloated containers in which applications can run

NOTE: Docker has already made the move away from Linux Containers (LXC) as it moves toward being a more OS independent product. As of version 0.9, Docker has started using its own container driver in favour of LXC.

Anyway, in a Docker world, a single underlying OS provides all the HW magic, scheduling, memory management and process goodness. Docker then allows us to install multiple apps on top of that single OS. Each app lives within a Docker container. These containers are lightweight, fast, and SIMPLE! Simpler than VM’s, faster than VM’s, and more lightweight than VM’s! And who wouldn’t want that!

I know this might be new to some, so forgive me for repeating the explanation (it may be useful for some)… Basically, Docker allows us to install multiple apps on top of a single kernel/OS (single OS license, single patching schedule, single mgt…). Each app runs within its own container. And if you know FreeBSD Jails, or Solaris Zones, you’ve a fairly good idea of what Docker containers are. It’s kind of like implementing user-space within containers. Each container is fully isolated and, effectively has it’s own process space, filesystem, memory and network… But it’s not a full VM!

So if you want three apps installing – let’s say Apache, MongoDB, and MySQL – just fire up three containers. One for Apache, one for MongoDB, and one for MySQL. And these containers are simpler, faster and lighter weight than having to spin up and hand-feed three full virtual machines!

It’s Early Days, But…..

Let’s not forget though, it’s early days for Docker. Legacy solutions like VMware ESXi, and other traditional Type 1 hypervisors, have been around for waaaaaay longer! So they’ve naturally got far richer owners toolsets and features. Things like VMware vMotion are awesome. And Docker’s not there, yet!

Also, Docker is really only a solid solution on Linux at the moment. But they’ve aspirations of being far more than that. It’s one of the major reasons they ditched Linux Containers (LXC)… and you can already get Docker solutions for Mac OSX and Windows – though they’re not anywhere near what Docker is on Linux… yet.

But the race is on. Docker is developing at a rapid pace, and it has some pretty massive players using it and throwing code at it. So watch out VMware, Docker is coming for you, and it doesn’t sleep!

Summary

The beauty of Docker is in its simplicity. And as nature proves – the simplest most efficient solutions usually win.

Obviously, for the time being, Docker will sit alongside traditional virtualization solutions like VMware and KVM. But because of it’s lightweight nature and simpler management etc, it will slowly start eating into that legacy virtual machine environment. Savvy people will be looking to move to Docker and Docker-type solutions more and more.

Will Docker, or containers in general, ever take over the world and fully replace traditional virtual machines? Hmmmmm may be not, but I certainly wouldn’t bet against it!

And I know that the VM world has it’s fanbois and intravenous kool-aid takers, but surely nobody expects the VM to be around forever! Nothing lasts forever! So it’s “farewell VM, it’s been nice knowing you. But move on over Grandpa, you’re starting to smell!“.

The big question now is…. who buys Docker? And how does the Open Source nature (Apache 2 license) of the code/project affect a potential acquisition?

Check out my Linux and Storage training courses over at Pluralsight. I seriously think they’re seriously awesome! Seriously! 😀

4 thoughts on “VM Sprawl – Docker to the Rescue!

  1. Pingback: Cloud Without Virtualization

  2. Jonny

    Are you sure Docker is ready to replace VMs? From what I have watched and read it is:
    “not recommend for long running services. The intended use is for a one shot container you build it and you kill it.”
    I tried to use Docker this week to create a Gitlab container requiring SSH & HTTP/S exposed but SSH is already listening on the host. On a VM I can choose a bridged interface and use an IP address on the external network as if the VM was on that external network. I’d like to achieve the same with Docker but I have not be able to find out how to do this. Maybe you could write an article on using Docker as if it was a VM ?
    Also long running processes/services are a bit problematic requiring supervisord or similar. Any ideas or feedback and I will be grateful.

  3. Pingback: Containers – Driving force or Delusion? | nothingbutcloud

  4. Scott Bamford

    Docker sounds like got lots of potential and I could see its adoption in the growing SME private cloud space being an important proving ground of the concept.

    From my time working on BSD kernels I’ve always felt the idea of “virtual machines” each running their own operating systems was abstracting a layer too low in the execution stack.

    If the kernel subsystems were abstracted along with a virtualised file system and each executable was able to express the subsystems it required and the OS-style for each one then many apps could avoid the need for an OS userland and be scheduled alongside full VMs by the host. Problem is that complete virtualisation of applications in this style would require standardisation of in-kernel communication APIs between kernel subsystems for all participating operating systems.

    The low hanging fruit to prove the theory if combined with JIT compilation would be to run Java JVM and .NET IL virtual machines as first-class OS-less executables too as these are already single app virtual machines of a sort.

    Sadly as there are (rightly) very few networked service processes written entirely in these environments it would be a pretty useless fruit to have picked (outside the mobile space that is). I think therefore technologies like Docker will have to normalise the idea of apps without full OS-userlands before we’ll see anyone give these existing app based VMs that sort of attention.

    P.S. not being copy-left means commercialisation after acquisition would probably be much easier than with a GPL based product with multiple contributors, so hopefully it won’t be a barrier to its development.

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You can add images to your comment by clicking here.