Author Archives: Nigel Poulton

Integrating Docker with Devops Automated Workflows

A few weeks ago I released a lightning fast (~1 hour) video course over at Pluralsight showing how to integrate Dockerized apps into a CI/CD automated workflow using some cool new features on Docker Hub.

Docker’s hot right. And we’ll be royally shafted if we don’t bring it under the control of our existing operational processes and the likes!  Think about it….. who wants tens/hundreds/thousands of Dockerized apps floating around in their estate unchecked? Not me!

Yes Docker’s the bizzo, yes Docker’s important, yes Docker’s blah blah blah. But unless we want a repeat of VM Sprawl and the headaches that brought,we need to get our act together over the reality of Docker and Dockerized apps.

Anyway…. the course takes a small web app + simple test, and shows how to push it through CI tests, integration with Docker Hub (build triggers, webhooks etc…..), and pushing to the world on Amazon Web Services!  All automated… and you’ll learn it all in a single hour! Sounds good to me!

Seriously….. if you don’t already have Dockerized apps in your estate:

  1. Go check again…. you actually might! Remember when we thought nobody was using AWS and then we learned about the whole shadow-IT thing…. Let’s not be caught sleeping at the wheel again.
  2. You will soon!  Seriously…. ignoring Docker aint gonna make it go away. So take a leaf out of the Scouting for Dummies book…. be prepared!

Anyway… that’s about it really… it’s a short fun course. And if you’re too tight-fisted to have a Pluralsight subscription, get one! they’ve always got a free 10-day trial going.

Production in the Cloud

This is my “Production in the Cloud” presentation form the recent TECHUNPLUGGED event in Amsterdam.

The skinny: It’s my opinion that public cloud services such as Amazon Web Services (AWS) and Microsoft Azure are ready for many enterprise production workloads. More than most of us think.

Sure…… there’ll be cases where they’re not a fit. But for those cases that are a fit….. you’d be hard pressed to find a more highly available and more world-class infrastructure to deploy on.

A few quick questions:

  • Do any of us really believe we can build better data centres than Amazon and Microsoft?
  • Do any of us really think we’re better at security than Amazon and Microsoft?
  • Do any of us really think we can build more highly available infrastructure than Amazon and Microsoft?

If you answered yes to any of the above…. then ask this final question…. can you do it at anywhere near the short term cost?

SaveTheTree – Docker in enterprises

First up…. this has nothing to do with saving trees – at least of the botanical variety. This has everything to do with Docker in the enterprise!


So… I spun up some new Docker hosts the other day… and it wasn’t long before I needed my trusty old friend `docker images –tree`. Well what was my horror when I got bitch-slapped with this:

npoulton@ip-10-0-0-90:/home/ubuntu$ sudo docker images --tree
 flag provided but not defined: --tree
 See 'docker images --help'.

Basically the `–tree` flag’s been pulled from the code! And yes, I know it’s been throwing “Warning: ‘–tree’ is deprecated” warnings at me since forever. I just never thought they’d actually go through with it.

And you know what right… I know it’s just a piece of software we’re talking about here.. but I’m seriously mortified by this. I don’t think I’ve ever had a more poignant lesson that it’s the litle things that make a big difference. Such a tiny command, that was so insanely powerful for Docker image management.

Enterprise Impact

Anyway…. what’s this all got to do with Docker in the enterprise?

Well…. I’ve spent enough time working big enterprises and financial services orgs that I know the odd ting about what gets signed off into production in these organizations and what doesn’t. So stick with me for a sec here…

Traditional enterprises – especially government, financial services etc – are as anal as the best of them when it comes to signing off code and services into production. Hell some of them still roll their own Linux kernels, not to mention still run stuff on AIX and pay through the teeth for EMC storage coz it makes them feel warm and fuzzy. Bottom line…. they soil their pants over every new thing they allow in to production.

So what I’m saying is….. if I was still at one of these types of orgs lobbying to get Docker signed off into production… I’d have taken the removal of `docker images –tree` as a steel-toe-capped kick in the old meat and two veg!

Why? Because now my ability to perform basic and vital image management tasks has become a whole lot harder. And the idea of running the folloing instead is just insane!

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock nate/dockviz images -t

Now I’ve no personal issue with Nate Jones and his actually quite cool little image. But the thought that I might be able to run code like that – spin up a random container from some guy called Nate who I’ve never met – on production systems is just mind blowing!

I’m sure doing this kind of stuff is done all the time in cool hipster companies and the likes – and I’m totally cool with that. But it’s absolutely not done in rusty old enterprises with the kind of big fat wallets that I’m sure Docker would love to help thin out.

So what I’m saying to Docker is….. and I say this with the deepest respect to those involved with the Docker project (props to you all for the genuinely awesome work you do)…. but please add the `–tree` option back. And keep good image and container management capabilities within the trusted core Docker codebase. The code that folks like me are no doubt trying to champion into production environments all over the world!

#SaveTheTree 😀

Learn Docker this Summer – for FREE!

Seriously… I can’t think of a better thing to do for your career this summer, than learning Docker!

Docker Learner

Well….. my Docker Deep Dive course (don’t be put off by the scary title) is part of select group of online video courses that make up Camp Pluralsight – a bunch of video training courses that are free during summer 2015.  A cracking way enhancing your career prospects and learning cool new tech at the same time.

Seriously…. if you’re not up to speed with Docker, this course is for you! I genuinely think it’s the best way to get up to speed – I mean I honestly did everything I could to make this the ultimate Docker learning experience.

Anyway… pop on over to Pluralsight where the course is available to watch now…  The containers are coming… don’t be caught sleeping!

PS.  Below is some feedback I’ve had via Twitter – notice the comments that say how good it is for newbies!

Docker course praise

Why you should learn to program in Go…

Huh…. how come an infrastructure guy like me just produced a Go programming course for Pluralsight?

I’ll keep this brief… but I think Go is gonna be mahoosively important… I expect Go skills to be right up there as one of the most important skills for IT folks to have in the near future.

Anyway… three quick things to say.

First up… I started my IT life as FoxPro programmer back in ~1998. In fact I still hold a grudge against Microsoft for what they did to it. Anyway, my point is, I’m partial to a small amount of coding and thought it would be fun to create a course on the fundamentals of Go 😉

Second up… Oh – my – goodness! Go is like an amazing language. And you know what… I reckon poised to take over the world. Now I’m not about to say that the Linux kernel is gonna get re-written in Go.  But seriously… go check out the major infrastructure projects that are being written in Go these days! Just to name a few – Docker, most of the CoreOS stuff like etcd, fleet…, Google’s open sourced Kubernetes!

Whoooaaa! These are like the who’s who of game-changing new infrastructure technologies! Well guess what… they’re written mostly in Go. So I figured, if anyone wants to be really good at any of these techs, knowing Go would be massively helpful.

Third up… a ton of infra people are looking to acquire good plots of land in the new DevOps world. Well I reckon Go will be a cracking language to have in your suitcase if you’re planning the move to the brave new DevOps world.

OK… so what to expect from the course?

The key is in the word “fundamentals”. My aim was to give a good solid intro to most of the features of Go. Check out the course outline… but it’s basically – variables, functions, conditionals, loops, concurrency…. The idea being…. you’ll get enough of a theory + hands-on intro to take your interest further.

go TOC

That’s it really. Go check it out! As always, there’s a free trial that you can sign-up for in case you don’t wanna part with your hard-earned cash on a promise I made here – sign-up for the trial and then cancel if you don’t like what you see!

Linux + Containers + Go = 42

Containers: Resistance is Futile!

In April 2015 I delivered this talk at the inaugural TECH.UNPLUGGED event in London.

The presentation’s aimed at getting enterprises ready for the container revolution.

You can also check out the other talks from the event by checking out the TECH.UNPLUGGED YouTube channel

And……. if you’re wanting to get yourself and your enterprise ready for containers… check out my awesome Docker courses over at Pluralsight – and don’t forget to check out any tiral offers they have!

Learn Docker Swarm – Native Docker Clustering

So continuing my work around Docker and Linux containers, a week or two ago I published another Docker related course on Pluralsight.  This one’s titled “First Look: Native Docker Clustering“…..

The title kind of gives it away – it’ll get viewers up to speed with running a small clustered Docker infrastructure using Docker Swarm.  Now then, this course is only for people who want to be ahead of the curve….. If that’s not you, then you don’t need to read any further.

Wanna be ahead of the curve!

Still here? Magic.

Well I’ve not got a lot to say on this, other than I’m convinced that tomorrows infrastructure will be built around containers…. And a massive part of this future infrastructure is gonna be automation and orchestration etc. And clustering will be central.  I think we’re already at the point where core container technologies are understood and being deployed in production.  So clustering and orchestration etc are next….

Probably at scale we’re gonna be looking at things like Kubernetes and Mesos etc. But for out-of-the-box/batteries-included clustering, Docker Swarm is gonna be a good clustering solution.

So if you’re wanting to get into containers and position yourself well for the infrastructure revolution, get yourself up to speed with containers and orchestration!

In that space, Docker’s a good place to start.  But its not the only container technology out there – Rocket from the guys at CoreOS is great too (though very different).  Then there’s a whole bunch of next gen orchestration and clustering technologies like Kubernetes and Mesos etc.  All gonna be massively important in the very near future.  But before mastering those, you need and understanding of containers and be cool with them first.

Still need to learn Docker?

If you need to learn Docker I highly recommend my Docker Deep Dive course, also available on Pluralsight.  It’s a soup-to-nuts course that starts at the very bottom and explains and demos all core Docker features in detail, taking you beyond intermediate level.  If you don’t know Docker yet, I genuinely believe this is *the* best place to learn it.

blog image

I’d also recommend my Docker Swarm course.  Both courses are there for people who wanna play and work with awesome technologies that are changing the face of infrastructure IT.

Thanks for reading, and let’s go change the world!

Here come the *real* server Operating Systems…

The days of server and desktop Operating Systems looking and feeling the same are gone – those are the olden days!

We’re being invaded by a new breed of *proper* server Operating Systems.  These guys are stripped down, hardened, tuned and optimised for server workloads. They’ve got none of the extra crap needed for things like a slick desktop experience. No! These guys are designed and built to live in the data centre and the cloud and they look a lot different to their frilly desktop siblings.

Following in the footsteps of Android

Think about it like this…. Android is Linux. It runs a Linux kernel. But we accept that it’s waaaaay different to any Linux we run on a desktop or server. We accept that it’s highly tuned to run well on mobile devices, because…… mobile devices and the things we do with them are totally different to desktops!

Well guess what…… the same holds true for servers….. the things we do with servers are totally different to the things we do with desktops!

So it should only be natural that our server Operating Systems hugely differ from our desktop Operating Systems.

Let’s look at an example of one…..


Take a look at RancherOS announced a couple days ago (24th Feb 2015)…. this is a server OS in every respect.

NOTE: RancherOS is only one example. I’m also a fan of CoreOS. But things like Project Atomic from Red Hat and Snappy Ubuntu Core from Canonical are heading in this direction too.


For starters…. the RancherOS ISO is an impressive 20MB – yes that’s MB and not GB! Talk about a small attack surface, and small code-base that requires patching… I’m guessing there’s no unnecessary code in their – definitely no pretty GUI! All good stuff for a server OS.

Also…. pretty much everything on RancherOS runs inside a Docker container. And we’re not just talking about applications. We’re talking about system services too! Stuff like syslog, and ntpd all run inside of their own Docker containers.

If you want to learn Docker… check out my Docker Deep Dive course available on Pluralsight. You can even sign-up for a free 10 day trial that will allow you to watch the entire course!

That means that if you log on to a RancherOS box and list the running processes, you’ll see a bunch of kernel threads dressed up as processes (that’s normal) but you won’t see much more than that. There’s no init or systemd…. PID1 will be a special Docker instance for running system processes as containers.  So as well as no systemd,  there’s also no package manager, no GUI, no unnecessary fluff.  Basically, it looks and feels nothing like desktop Linux!


In my opinion….. this is all good stuff, and I love innovative stuff like this!

And it makes total sense…. servers are totally different beasts to desktop machines. I mean even Microsoft *started* to grock when it shipped Server Core. And well done to MS for finally shipping a server OS without a GUI. But the stuff going on in the Linux world is so far beyond that. We’re seeing massive changes for the cloud, automation, and container-based workloads.

The future’s bright… the future’s a new breed of minimalist, purpose-built, server Operating Systems!

NOTE: I’m not saying using Docker as PID1 or any of this stuff is production ready yet….. what I am saying though… is I love this kind of stuff and can see this kind of stuff being the future!

Quick Question on VSAN vs XtremIO Architectures

This is just a really quick one……

I noticed that upgrading an existing VSAN to the newly announced VSAN 6.0 is a non-disruptive rolling upgrade.

Fair enough, so it should be right?

Well…… part of the upgrade requires what sounds like a fairly significant underlying on-disk layout change.  Basically, the upgrade to VSAN 6.0 updates the underlying layout to a new format based on VirstoFS technology.  Sounds major to me!

So with that in mind…. I think an online rolling upgrade is fairly impressive.  Well done to the VSAN team.

So If VSAN Can Do It Why Can’t Others?

The question then begs….. if VSAN can upgrade the underlying disk layout on the fly as part of an online upgrade, then why can’t XtremIO (or any other modern storage technology for that matter)?!?!?!

Is the VSAN architecture superior?

Now I get that there are different use-cases for storage technologies, and that there’s not a one-size-fits-all architecture. But on the topic of non-disruptive upgrades… is VSAN a superior architecture?

Am I Missing Something?

May be I’m missing something.  May be the on-disk changes that VSAN is making are different and less fundamental than those recently made by XtremIO?

Just a curious guy asking a genuine question.

Comments welcome (please state your vendor affiliation).

Docker on Windows – Some Insight

So last week I had a cracking and informative conversation on Twitter about native Docker support on the next version of Windows Server. So I thought it only right to scribble down the good stuff here for anyone who didn’t get a chance to listen in and get involved.

It all started when Stu Miniman from Wikibon engaged with me over my previous post ESXi vs Hyper-V – Could Docker Support Be Significant. And to cut a long story short, Stu looped in Nick Weaver (formerly EMC, now at Intel), and Nick looped in Solomon Hykes (founder and CTO of Docker Inc). Anyway…. a bunch of stuff was discussed and the following is what interested me the most.

ALERT: Looks like the guys at MS won’t be shipping Windows Server vNEXT for a vLONG time – thanks to Ewan Leith for pointing that out to me.

On the Topic of Native Docker Support on Windows

First up OK….. when I’m talking about native Docker support on Windows Server, I’m talking about Windows running the Docker daemon – the core Docker engine. Meaning we’ll be able to take Windows Server and launch Docker containers on it, all natively. That’s what I’m fully expecting Microsoft to ship at some point on the next version of Windows Server – whenever that happens to be!!!!

NOTE: Now of course…. these containers will be leveraging the Windows kernel, so Linux containers won’t run on a Windows Server running Docker. Let’s be clear about this, Docker containers share access to the kernel of the host machine they’re running on, meaning Windows apps/containers will only run on Windows Docker hosts, and Linux apps/containers will only run on Linux Docker hosts.

Anyway….. on to the interesting stuff….

Porting Docker to Windows is Non-trivial

First up, Solomon states that porting Docker to Windows is non-trivial – obviously the Windows and Linux kernels are very different. But he does point out that the core abstractions of processes, files, networking etc are all the same.


He also points out that the Windows guys have the luxury of starting from scratch, but also the extra requirements of the Windows Registry.  No wonder Windows Server vNEXT has been pushed back to 2016….


Copy on Write and Union Mount Filesystems

Solomon then says that the copy-on-write/union mount filesystem approach that Docker images and containers currently rely on will be very different on Windows.


Obviously Windows isn’t anywhere near as rich as Linux when it comes to filesystems and union filesystems – my words not his. But, he does point out that the Docker storage backend is already pluggable. For example, a bunch of different back-end filesystems can be used by Docker. Some of these include AUFS, devicemapper, BTRFS, and recently Overlayfs.

So at the core of Docker 1.x is the need for a CoW filesystem/union mount backend – the way images are built, and the way containers are launched is all built on top of CoW and union mounts.

NOTE: I cover this in my Docker Deep Dive course. In particular in Module 6 – A Closer Look at Images and Containers.

However, Solomon stated that *changing this requirement* will help. And when I asked, he said that the v2 Image format will be taking a step in that direction.


The Docker Execution Driver

Now some other Linux goodness that’s core stuff for Docker containers are kernel namespaces, cgroups, and capabilities. The short and skinny of these is as follows –

  • Namespaces let us carve up things like the process tree, the filesystem, networking etc so that each container gets its own unique and isolated view of each – basically making a container think its got PID 0 and the root filesystem (/) and not knowing about other containers on the system….
  • croups let us control resource utilisation for containers
  • Capabilities let us get very granualr with container privileges

All of these are vital to a solid container system like Docker, and all are native in the Linux kernel.  So the question begs…. does the Windows kernel have anything similar up it’s sleeve?

Well your guess is as good as mine…. and apparently as good as Solomon’s.


There’s a bunch of talk in social media about whether things like App-V, ThinApp, Drawbridge etc might be able to provide some of this functionality.  But it’s all speculation at the moment.

Microsoft and Open-source

Now then, on the topic of stuff like namespaces, cgroups and capabilities…… The way that Docker leverages these in a Linux environment is through a pluggable component called the execution driver.

And when Docker started out in life, it used LXC as its execution driver. However, this wasn’t ideal, as it was central to providing Docker with access to vital kernel features, and it was essentially borrowed technology. So in order to give themselves more control over core Docker components, Docker Inc chose to implement their own execution driver called libcontainer.

NOTE: Again, this is all covered in my Docker Deep Dive course

Anyway, the fact that Solomon says most of the work in this area is on the shoulders of the Windows guys, suggests – at least to me – more of an LXC based approach to the execution driver on the Windows platform (where Docker Inc are at least in part reliant on others – Microsoft in this case).

Also….. and this is nothing short of rash unsupported speculation from me here….. but if it were the case that MS own and not keep the native execution driver on the Windows platform closed-source…. would it give Microsoft the opportunity to “take the ball home” one they’ve sucked what they can from the Docker ecosystem????  But that’s just scandalous speculation from me – don’tread anything into it!

Docker and Open-source

Anyway…. the conversation sparked me to ask Solomon whether or not the Docker execution driver and storage driver for Windows would be open source – it’s well known that the guys at Docker are passionate about doing everything in the open. Solomon’s reply was that they won’t merge anything into Docker that isn’t open source. Cool!


But since the conversation, I’ve wondered…… does that mean the code will be open sourced…. or does it mean that they won’t be, and those particular pluggable components won’t be “merged”. Hmmmmm????

All in all a great informative conversation! I hope my thoughts and opinions haven’t munged the facts with too much speculation and outright fiction 😀

A massive thanks to Stu, Nick and Solomon for part of the conversation. These are the types of guys I could spend all day talking to and learning from!

Comments welcome.