This is the first in a short series of posts I’m doing on NFSv4. They’re kind of educational, a bit of an NFS theory tutorial if you will, with a bit of opinion thrown in to the mix.
BTW If you’ve arrived here by searching for NFS and were expecting Need For Speed, pack your bags and be on your way. This site is not about toys or games, it’s an adult-only technology site where we get waist deep in the technologies that make the world turn around. This post is about the real NFS… Network File System.
I’ve engaged in a few NFS related conversations on Twitter recently, and had a couple of requests to do a deep dive on NFSv4. So here we go…..
NFSv4 In a Nutshell
Below are what I consider the major technical points of interest in NFSv4 -
- Stateful protocol (earlier versions were essentially stateless)
- Access Control Lists – similar to Windows ACLs
- Mandated strong security
- COMPOUND RPC’s
- Delegations and client-side caching
- TCP and a well defined port
- Parallel NFS a.k.a. pNFS implemented as part of NFSv4.1
Over the next few articles we’ll cover all of the above. As for today, lets do a brief history and cover off COMPOUND RPCs.
Whistle-stop NFS history
NFS is old. Originally designed and implemented in the 80’s by the late and great Sun Microsystems. The first version available outside of Sun was version 2 (NFSv2) and it was stateless and operation over UDP with no requirements for decent security – did security exist in IT back then? Not great for use over the internet, but then back in the day I suppose tinternet wasn’t what it is now.
Version three (NFSv3) came in the mid 90’s and was officially developed and owned by Sun, but was clearly outgrowing Sun and by the time version 4 (NFSv4) came along in 2000 Sun had nobly (well, let’s speak well of the deceased) handed development and change control over to the IETF.
These days all of the NAS big-boys like NetApp, EMC, IBM, BlueArc, Isilon, Panasas… as well as the OS boys like RedHat and Microsoft are all over it like a rash! In fact speaking of Microsoft, it’s fair to say that Microsoft’s proprietary CIFS/SMB protocol had significant influence over the development of NFSv4. More on that later in the series of posts.
Crazy out there thought: As NFS goes from strength to strength I have to wonder how long it will be before Microsoft drops CIFS and fully embraces NFS!!?? Of course this is pure speculation, but Microsoft already has a decent NFS client for Windows, and the University of Michigan Center for Information Technology Integration (CITI) recently released a pNFS driver for Windows. One has to wonder how much longer Microsoft can justify funding development of CIFS in the face of a
betterperfectly good alternative….?? :-S
Anyway, on the technical stuff….
NFS and chatty RPCs
NFS runs over ONC RPC (Open Network Computing Remote Procedure Call).
While talking about ONC RPC I have to point out the following. NFS has been around for donkeys years, and has been running over ONC RPC for ages. Funny thing is, although ONC RPC was proposed in 1995, it is still not an official Internet standard! In 2009, 14 years after it was first proposed, it was finally accepted as a DRAFT Standard! Wonder how long before it becomes a fully fledged Internet Standard :-S And you thought the guys peddling FCoE were taking a risk by pushing half-baked standards
Previous versions of NFS were chatty. If you thought Chuck Hollis had verbal diarrhea, he’s got nothing on previous versions of NFS! Basically,all actions in NFSv2 and NFSv3 were short sharp RPC’s, and the client and server would ping-pong multiples of these short RPCs to each other. The below diagram attempts to show this behaviour -
Apologies for the crappness of the diagram, but you get the point. In order to mount and read a portion of a file in previous versions of NFS could take 20+ exchanges – search the internet for NFS protocol sequence, you’ll find some better pictures.
The above is of course not the best, none of us like chatty network protocols, and the performance effects are compounded (pun intended) in high latency networks.
Clearly NFS needed to move with the times and shape up.
For the record even CIFS in SMB2 can compound requests and is therefore less chatty than it used to be. Although CIFS only started compounding in SMB2 ~2006.
NFSv4 and COMPOUND RPCs
So, COMPOUND RPCs to the rescue!
BTW I’m capitalising COMPOUND because most of the official looking docs on the subject seem to do that (such as here). So in an attempt to try and look like an expert I’ll follow their lead - will no doubt backfire on me
The principle is simple. Clients can batch up operations and send them as fewer RPC’s. The same goes with responses from the server, operation responses get batched up too. Less chatty. Marvellous!
Keeping with the lingo again, it seems that NFSv4 and onward refers to actions as operations, whereas versions prior to v4 referred to operations as actions :-D Anyway NFSv4 we refer to Operations.
If the server encounters an error half way through executing the operations in the COMPOUND RPC it will not process and execute the remainder of the operations, instead it will return the results of all successful operations as well as the appropriate error to the client.
The above allows NFSv4 to behave better on today’s data centre and internet networks. Wanna try mounting NFSv2 of v3 exports over the internet. NFSv4 might be doable though. In fact, some of the good stuff in NFSv4 might just make it ideal for the internet, may be they should rename it to….. hmmmm…… something like Common Internet File System….
One final point since I’ve gone and mentioned mounting. Changes have happened to the mount protocol in NFSv4 too, kind of related to RPCs actually but Match of the Day is about to start so I’ll touch on that in a later post.
Courteous comments welcome.
BTW, I’ve been running a poll on the sidebar of the website asking which protocol will be the dominant storage protocol in 5 – 10 years; CIFS, FC, FCoE, iSCSI, NFS, None of the above… Please cast your vote.
A big thanks to EMC and NetApp. I made a call out recently on Twitter for NFS related info. I suppose it should be no surprise that NetApp and EMC were the ones to respond. Cheers!