Docker on Windows – Some Insight

By | February 2, 2015

So last week I had a cracking and informative conversation on Twitter about native Docker support on the next version of Windows Server. So I thought it only right to scribble down the good stuff here for anyone who didn’t get a chance to listen in and get involved.

It all started when Stu Miniman from Wikibon engaged with me over my previous post ESXi vs Hyper-V – Could Docker Support Be Significant. And to cut a long story short, Stu looped in Nick Weaver (formerly EMC, now at Intel), and Nick looped in Solomon Hykes (founder and CTO of Docker Inc). Anyway…. a bunch of stuff was discussed and the following is what interested me the most.

ALERT: Looks like the guys at MS won’t be shipping Windows Server vNEXT for a vLONG time – thanks to Ewan Leith for pointing that out to me.

On the Topic of Native Docker Support on Windows

First up OK….. when I’m talking about native Docker support on Windows Server, I’m talking about Windows running the Docker daemon – the core Docker engine. Meaning we’ll be able to take Windows Server and launch Docker containers on it, all natively. That’s what I’m fully expecting Microsoft to ship at some point on the next version of Windows Server – whenever that happens to be!!!!

NOTE: Now of course…. these containers will be leveraging the Windows kernel, so Linux containers won’t run on a Windows Server running Docker. Let’s be clear about this, Docker containers share access to the kernel of the host machine they’re running on, meaning Windows apps/containers will only run on Windows Docker hosts, and Linux apps/containers will only run on Linux Docker hosts.

Anyway….. on to the interesting stuff….

Porting Docker to Windows is Non-trivial

First up, Solomon states that porting Docker to Windows is non-trivial – obviously the Windows and Linux kernels are very different. But he does point out that the core abstractions of processes, files, networking etc are all the same.

docker-win-solomon-1

He also points out that the Windows guys have the luxury of starting from scratch, but also the extra requirements of the Windows Registry.  No wonder Windows Server vNEXT has been pushed back to 2016….

docker-win-solomon-2

Copy on Write and Union Mount Filesystems

Solomon then says that the copy-on-write/union mount filesystem approach that Docker images and containers currently rely on will be very different on Windows.

docker-win-solomon-3

Obviously Windows isn’t anywhere near as rich as Linux when it comes to filesystems and union filesystems – my words not his. But, he does point out that the Docker storage backend is already pluggable. For example, a bunch of different back-end filesystems can be used by Docker. Some of these include AUFS, devicemapper, BTRFS, and recently Overlayfs.

So at the core of Docker 1.x is the need for a CoW filesystem/union mount backend – the way images are built, and the way containers are launched is all built on top of CoW and union mounts.

NOTE: I cover this in my Docker Deep Dive course. In particular in Module 6 – A Closer Look at Images and Containers.

However, Solomon stated that *changing this requirement* will help. And when I asked, he said that the v2 Image format will be taking a step in that direction.

docker-win-solomon-4

The Docker Execution Driver

Now some other Linux goodness that’s core stuff for Docker containers are kernel namespaces, cgroups, and capabilities. The short and skinny of these is as follows –

  • Namespaces let us carve up things like the process tree, the filesystem, networking etc so that each container gets its own unique and isolated view of each – basically making a container think its got PID 0 and the root filesystem (/) and not knowing about other containers on the system….
  • croups let us control resource utilisation for containers
  • Capabilities let us get very granualr with container privileges

All of these are vital to a solid container system like Docker, and all are native in the Linux kernel.  So the question begs…. does the Windows kernel have anything similar up it’s sleeve?

Well your guess is as good as mine…. and apparently as good as Solomon’s.

docker-win-solomon-5

There’s a bunch of talk in social media about whether things like App-V, ThinApp, Drawbridge etc might be able to provide some of this functionality.  But it’s all speculation at the moment.

Microsoft and Open-source

Now then, on the topic of stuff like namespaces, cgroups and capabilities…… The way that Docker leverages these in a Linux environment is through a pluggable component called the execution driver.

And when Docker started out in life, it used LXC as its execution driver. However, this wasn’t ideal, as it was central to providing Docker with access to vital kernel features, and it was essentially borrowed technology. So in order to give themselves more control over core Docker components, Docker Inc chose to implement their own execution driver called libcontainer.

NOTE: Again, this is all covered in my Docker Deep Dive course

Anyway, the fact that Solomon says most of the work in this area is on the shoulders of the Windows guys, suggests – at least to me – more of an LXC based approach to the execution driver on the Windows platform (where Docker Inc are at least in part reliant on others – Microsoft in this case).

Also….. and this is nothing short of rash unsupported speculation from me here….. but if it were the case that MS own and not keep the native execution driver on the Windows platform closed-source…. would it give Microsoft the opportunity to “take the ball home” one they’ve sucked what they can from the Docker ecosystem????  But that’s just scandalous speculation from me – don’tread anything into it!

Docker and Open-source

Anyway…. the conversation sparked me to ask Solomon whether or not the Docker execution driver and storage driver for Windows would be open source – it’s well known that the guys at Docker are passionate about doing everything in the open. Solomon’s reply was that they won’t merge anything into Docker that isn’t open source. Cool!

docker-win-solomon-6

But since the conversation, I’ve wondered…… does that mean the code will be open sourced…. or does it mean that they won’t be, and those particular pluggable components won’t be “merged”. Hmmmmm????

All in all a great informative conversation! I hope my thoughts and opinions haven’t munged the facts with too much speculation and outright fiction 😀

A massive thanks to Stu, Nick and Solomon for part of the conversation. These are the types of guys I could spend all day talking to and learning from!

Comments welcome.

 

7 thoughts on “Docker on Windows – Some Insight

  1. Chris Evans

    Nigel,

    Here are a few other things to think about; windows uses dynamic linking (DLLs) for loadable modules. DLL Hell results because of the way Microsoft bodged the approach to user written DLLs. This is the Side by Side assembly (or WinSXS directory). Have a look at it some time on a system with multiple applications. It’s a mess. So how will Docker manage this?

  2. Scott Bamford

    Would be interesting to see if at any point a *BSD like binary emulation layer would be added to the Windows kernels to allow Linux containers to execute alongside Windows containers against a single kernel.

    The basic concept is to recognise the type of binary being executed and thereafter apply a mapping of syscalls for each process. Very low performance overhead.

    At a process level rather than container level I use this a lot on BSD and always saw a similar solution suitable for Microsoft if it ever needed to be Linux compatible. Blocking issues like lack of kernel support for fork() could be solved at the same time (it’s only a big issue to implement in user space, in kernel space its conceptually simple).

    How would only Windows Server offering single kernel execution of Windows and Linux containers make an impact on the future landscape?

    Links to explain the syscall *BSD emulation:
    http://www.netbsd.org/docs/compat.html
    http://www.freebsd.org/doc/en/articles/linux-emulation/freebsd-emulation.html

  3. Nigel Poulton Post author

    @scott

    The ability to run Linux containers on a Windows Server would absolutely be a game changer in the ESXi vs Hyper-V battle. But obviously baby steps first…. We need to see how good a job MS does with version 1 of native Docker support first. Though the recent rise of containers as the de facto standard for *cloud-native* scalable apps might push Microsoft’s hand to look at something like that – as it stands they look to be falling behind in this race (at least from where I stand).

    Not sure I understand your last question “How would only Windows Server offering single kernel execution of Windows and Linux containers make an impact on the future landscape?”

    My initial thoughts behind this post were relative to Hyper-V vs EXSi. Clearly ESXi can’t execute Windows or Linux containers directly on the VMkernel – they can only execute inside of Windows and Linux VMs. If Hyper-V offers everything ESXi does (i.e. Windows and Linux VMs) *plus* native execution of Windows containers bare back on the Windows kernel (i.e. not inside a VM) then that gives it an advantage over and above ESXi when choosing a hypervisor. Though of course, if Hyper-V also had some form of binary emulation layer that allowed Linux containers to execute on Hyper-V outside of a VM, that may be even more of an advantage. I say *may* be as this is all speculation and I’ve zero idea whether a binary emulation layer would be a lot more lightweight than a VM (I assume yes).

    Just a shame MS have pushed Windows Server vNEXT back to 2016. It all seems a bit far off at the moment!

  4. Scott Bamford

    @Nigel

    Thanks for sharing your insights in the response. You did answer my question and its good to get some input from someone who knows VM and container technology so well.

    Binary emulation between two Unix-like OSs normally has a performance impact of <1% of the syscall's execution time (i.e. unnoticeable) as the only tasks done are to realign or reorder arguments in registers before making the call. However Windows cannot be considered a Unix-like OS and since these binary emulation layers first became heavily used 15 or so years ago:

    1. Linux has also grown is number of OS specific syscall's
    2. all significant efforts to standardise new syscalls across Unix-like OSs (especially Linux) have all but stopped and Linux has become completely in the Unix-like OS space.

    As such the effort to support native versions of these new syscalls, never mind the emulation layer, may itself now prevent anyone considering it as a serious solution. Interestingly due to its heritage, binary emulation of Linux binaries/containers on OSX would be much simpler – but its hard to think of a real world use for that.

    One way or another Microsoft in particular need to get a handle on efficiently running Linux and Windows Server based micro-VMs/containers/or executables if its to keep control of its own costs with the constantly increasing Azure growth. Doing this in a way that gives strong marketable benefits to its Windows Server products in private cloud deployments would make a lot of commercial sense.

    My own fear is that Microsoft's focus and timescales on Windows Server vNext, .NET vNext, and similar vNext efforts alongside the currently drive for openness and open source – while all individually good things – could leave those looking for steady incremental change feeling disappointed in the Microsoft stack for the next few years, and unwilling to make the API leaps required to make the best of the platform even once available.

  5. Nigel Poulton Post author

    @scott.

    Thanks for your in-depth input Scott. I think your final point about release cycles of Microsoft products is very interesting. Their approach is starting to feel extremely *last year*!

  6. Angel

    Nigel, thanks for sharing this with us.

    Obviously I’m pretty sure we’ll get Docker on Windows, if Parallels was able to do Windows containers with Virtuozzo, why can’t they? My only concern is patching, which was very critical with Virtuozzo…hope it isn’t now, it’s a while since I tested it.

    Regards
    Angel

  7. Brad

    Have you tried Spoon? Very similar functionality to Docker with Application Layering. However… I’ll be the first admit it has a long way to go and not open-source. It does give us some perspective though on what we can expect.

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You can add images to your comment by clicking here.