Comments on: Xsigo – Try it out, I dare you! http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/#utm_source=feed&utm_medium=feed&utm_campaign=feed with nigel poulton Tue, 20 Dec 2011 19:37:54 +0000 http://wordpress.org/?v=2.8.6 hourly 1 By: Some Xsigo Links2vcps and a Truck | 2vcps and a Truck http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-778 Some Xsigo Links2vcps and a Truck | 2vcps and a Truck Thu, 24 Jun 2010 16:34:22 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-778 [...] Posts Xsigo – Try it out, I dare you! - Nigel Poulton (Awesome review and great [...] [...] Posts Xsigo – Try it out, I dare you! – Nigel Poulton (Awesome review and great [...]

]]> By: Cam Ford http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-594 Cam Ford Thu, 19 Nov 2009 18:12:10 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-594 Ethernet and RDMA - Brad, thanks for reminding me to comment. For those of you interested in RDMA capabilities, the Infiniband Trade Association (IBTA) is currently in the process of standardizing RDMA on Ethernet, called RoCEE (pronounced Rocky....not my favorite...) which stands for RDMA over Converged Enhanced Ethernet.  Much like its little brother, FCoE, this standard will directly map RDMA transport frames onto Ethernet and leverage the emerging CEE capabilities for reliable physical layer transport. This technology already works today on existing Mellanox HCAs.  Some of you may know that Mellanox has an ASIC (ConnectX) that runs both 10GbE and 10, 20, and 40Gb IB on the same chip.  It also does RDMA over Ethernet (demoed at Interop and SC09, but not yet released). Its not so surprising then that Mellanox was the 1st company to announce a 40GbE NIC.....as it is basically the same 40Gb IB HCA that also supports Ethernet framing...although I believe it is a step of the ConnectX ASIC (called ConnectX-2).  The really cool part is that both 40Gb IB and 40GbE are going to use QSFP optics.....so the only difference between adn IB port and anEthernet port is the firmware running on the adapter......now this is true covergence. Here is the best part.....all IB Host drivers are OPEN SOURCE....let me say that again.....OPEN SOURCE.  Once you get RDMA on Ethernet, you now have full access to OPEN SOURCE drivers for RDMA, MPI, SRP, ISER, IP, etc. and you can map just about any driver for any application on top of it.  Imagine...no more vendor specific Hardware drivers for networking, storage, cluster protocols, etc..... Ethernet and RDMA – Brad, thanks for reminding me to comment.
For those of you interested in RDMA capabilities, the Infiniband Trade Association (IBTA) is currently in the process of standardizing RDMA on Ethernet, called RoCEE (pronounced Rocky….not my favorite…) which stands for RDMA over Converged Enhanced Ethernet.  Much like its little brother, FCoE, this standard will directly map RDMA transport frames onto Ethernet and leverage the emerging CEE capabilities for reliable physical layer transport.
This technology already works today on existing Mellanox HCAs.  Some of you may know that Mellanox has an ASIC (ConnectX) that runs both 10GbE and 10, 20, and 40Gb IB on the same chip.  It also does RDMA over Ethernet (demoed at Interop and SC09, but not yet released).
Its not so surprising then that Mellanox was the 1st company to announce a 40GbE NIC…..as it is basically the same 40Gb IB HCA that also supports Ethernet framing…although I believe it is a step of the ConnectX ASIC (called ConnectX-2).  The really cool part is that both 40Gb IB and 40GbE are going to use QSFP optics…..so the only difference between adn IB port and anEthernet port is the firmware running on the adapter……now this is true covergence.
Here is the best part…..all IB Host drivers are OPEN SOURCE….let me say that again…..OPEN SOURCE.  Once you get RDMA on Ethernet, you now have full access to OPEN SOURCE drivers for RDMA, MPI, SRP, ISER, IP, etc. and you can map just about any driver for any application on top of it.  Imagine…no more vendor specific Hardware drivers for networking, storage, cluster protocols, etc…..

]]> By: Cam Ford http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-591 Cam Ford Thu, 19 Nov 2009 16:11:41 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-591 Brad, You are absolutely correct.  Xsigo does change the way the datacenter is managed.  We greatly simplify the functions that are being done today by allowing the Server guys to dynamically create vNICs and vHBAs running on live servers anytime they want....and manage those resources in a really easy way. No longer will a Server admin be held hostage by a Networking group who takes weeks to months to pull a new cable to a server.  Same goes for storage. Equally has important is how we allow the DC architecture to be managed in the same logical topology that it is today.  The Xsigo approoach to IO Virtualization does not force the merging of Storage network management and datanetwork management.  We do not require VSANs and VLANs to overlay in the same switch.  We simply extend the IO bus from the server into an aggregation device (VP780 IO Director) where those resources can be virtualized, shared, and centrally managed.  NICs are still NICs, HBAs are still HBAs, etc....only everything is virtual and managed electronically instead of physically.  SAN management is fully independent from network management, etc.....allthough the mangemetn is actually done on teh same box. One more important point.  In a Xsigo environment, IT shops can virtualize their IO across HP, IBM, Dell, Hitachi, Fujitsu, SuperMicro, Verarri, and virtually anyone elses servers....in both Rack and Blade configurations.  No one is locked into a single vendor solution.  And while some folks have prematurely declared IB, FC, ATM, FICON, and numerous other technoloties dead......i will happily let you know that Xsigo has Cisco IB switches and Cisco IB HCAs happily running in our certification environment along side everyone elses gear in the same virtual IO environment. Brad,
You are absolutely correct.  Xsigo does change the way the datacenter is managed.  We greatly simplify the functions that are being done today by allowing the Server guys to dynamically create vNICs and vHBAs running on live servers anytime they want….and manage those resources in a really easy way.
No longer will a Server admin be held hostage by a Networking group who takes weeks to months to pull a new cable to a server.  Same goes for storage.
Equally has important is how we allow the DC architecture to be managed in the same logical topology that it is today.  The Xsigo approoach to IO Virtualization does not force the merging of Storage network management and datanetwork management.  We do not require VSANs and VLANs to overlay in the same switch.  We simply extend the IO bus from the server into an aggregation device (VP780 IO Director) where those resources can be virtualized, shared, and centrally managed.  NICs are still NICs, HBAs are still HBAs, etc….only everything is virtual and managed electronically instead of physically.  SAN management is fully independent from network management, etc…..allthough the mangemetn is actually done on teh same box.
One more important point.  In a Xsigo environment, IT shops can virtualize their IO across HP, IBM, Dell, Hitachi, Fujitsu, SuperMicro, Verarri, and virtually anyone elses servers….in both Rack and Blade configurations.  No one is locked into a single vendor solution.  And while some folks have prematurely declared IB, FC, ATM, FICON, and numerous other technoloties dead……i will happily let you know that Xsigo has Cisco IB switches and Cisco IB HCAs happily running in our certification environment along side everyone elses gear in the same virtual IO environment.

]]> By: Brad Hedlund http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-593 Brad Hedlund Thu, 19 Nov 2009 14:01:15 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-593 Looks like not every one understands that Ethernet is also capable of RDMA. Looks like not every one understands that Ethernet is also capable of RDMA.

]]> By: Amnon http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-592 Amnon Wed, 18 Nov 2009 21:05:59 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-592 NigelGreat post that gives some background to the Xsigo architecture.Some follow on questions:  Are the HBA and Ethernet modules standard off the shelf (I assume they are) and are they hot pluggable?For the HBAs/Ethernet controller - can any HBA be used or does Xsigo limit to specific vendors? Can the customer use an 8G FC HBA from Qlogic/Emulex?Thanks,Amnon NigelGreat post that gives some background to the Xsigo architecture.Some follow on questions:  Are the HBA and Ethernet modules standard off the shelf (I assume they are) and are they hot pluggable?For the HBAs/Ethernet controller – can any HBA be used or does Xsigo limit to specific vendors? Can the customer use an 8G FC HBA from Qlogic/Emulex?Thanks,Amnon

]]> By: Etherealmind http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-590 Etherealmind Wed, 18 Nov 2009 08:48:06 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-590 Looks like not everyone understands that Infiniband allows direct memory access between blades/brcicks. That is, using RDMA an application can write data directly to a memory location of another machine, at nanoseconds delay. Thie concept of sending data between servers using Ethernet is quaint and slow by comparison. Looks like not everyone understands that Infiniband allows direct memory access between blades/brcicks. That is, using RDMA an application can write data directly to a memory location of another machine, at nanoseconds delay. Thie concept of sending data between servers using Ethernet is quaint and slow by comparison.

]]> By: Ariel Cohen http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-589 Ariel Cohen Wed, 18 Nov 2009 03:02:38 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-589 Brad,   I already said twice that even vNIC-to-vNIC switching on the same port can be performed externally if desired and if the external switch supports it.   I don't see any "confusion" when it's performed on the vNIC module in the case where the vNICs terminate on the same port. It's standard practice for virtualized NICs regardless of whether they are on a virtual switch within a hypervisor, or a hardware virtualized NIC within the server (e.g. an SR-IOV NIC), or an externally virtualized hardware NIC (e.g. Xsigo). Again, cases where users want to do even this externally can be addressed with hairpin forwarding.   An I/O Director is a system providing shared virtualized I/O controllers to servers. Its primary function is very different from that of a switch regardless of whether the I/O Director is configured to perform some switching between vNICs or none at all.   Brad,
 
I already said twice that even vNIC-to-vNIC switching on the same port can be performed externally if desired and if the external switch supports it.
 
I don’t see any "confusion" when it’s performed on the vNIC module in the case where the vNICs terminate on the same port. It’s standard practice for virtualized NICs regardless of whether they are on a virtual switch within a hypervisor, or a hardware virtualized NIC within the server (e.g. an SR-IOV NIC), or an externally virtualized hardware NIC (e.g. Xsigo). Again, cases where users want to do even this externally can be addressed with hairpin forwarding.
 
An I/O Director is a system providing shared virtualized I/O controllers to servers. Its primary function is very different from that of a switch regardless of whether the I/O Director is configured to perform some switching between vNICs or none at all.
 

]]> By: Brad Hedlund http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-588 Brad Hedlund Wed, 18 Nov 2009 01:46:59 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-588 Ariel, you said: "vNIC-to-vNIC traffic on the same port is forwarded on the vNIC I/O module itself." That makes the "I/O Director" a network switch. You also said: "When the vNICs terminate on different Xsigo I/O module ports, traffic between them will be switched by the external Ethernet switches" This means is that some traffic might be handled by the Ethernet network, some might not, it all depends on how the I/O Director is configured. How is that not only changing the network management model, other than completly confusing it? Cheers, Brad Ariel, you said:
“vNIC-to-vNIC traffic on the same port is forwarded on the vNIC I/O module itself.” That makes the “I/O Director” a network switch.

You also said:
“When the vNICs terminate on different Xsigo I/O module ports, traffic between them will be switched by the external Ethernet switches”

This means is that some traffic might be handled by the Ethernet network, some might not, it all depends on how the I/O Director is configured. How is that not only changing the network management model, other than completly confusing it?

Cheers,
Brad

]]> By: Ariel Cohen http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-587 Ariel Cohen Wed, 18 Nov 2009 01:08:20 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-587 No, Brad. That is not what I said. In fact, there is no change to the network management model.   As to being a switch - you can perform the Ethernet switching in your favorite switches which are external to the I/O Director, and connected to the I/O Director Ethernet vNIC ports. Even the vNIC-to-vNIC switching on the same port can be done externally if desired when using switches which support hairpin forwarding.   No, Brad. That is not what I said. In fact, there is no change to the network management model.
 
As to being a switch – you can perform the Ethernet switching in your favorite switches which are external to the I/O Director, and connected to the I/O Director Ethernet vNIC ports. Even the vNIC-to-vNIC switching on the same port can be done externally if desired when using switches which support hairpin forwarding.
 

]]> By: Ilja Coolen http://blog.nigelpoulton.com/xsigo-try-it-out-i-dare-you/comment-page-1/#comment-586 Ilja Coolen Wed, 18 Nov 2009 00:02:27 +0000 http://blogs.rupturedmonkey.com/?p=606#comment-586 Thank you all for your responses. Very helpfull and enlightning. But one question goes unanswered. Maybe I wasn't clear enough (likely). vHBA and NPIV make sense to me. Now, I imagine there are restrictions as to how flexible NPIV exposure to one of the connected SAN fabrics is. I'll try to clarify my question by using some example ports. If I were to connect IB port 1 to a host, and created 4 vHBA's. I obviously would get 4 NPIV ports ready to be exposed to connected FC ports. Can these 4 NPIV ports be exposed to let's say 4 different FC ports and thereby 4 different fabrics? Or is there a fixed technical boundary were all NPIV ports on an IB port have to be assigned to the same FC port? Sorry, another question just popped up? Might be a stupid one, but I'll post it anyway. The vHBA appears like any regular HBA to the OS, like for example VMware. Within VMware I can also create virtual HBA. Knowing that it would make no sense for real world applications, but would it technically be possible to stack the VMware Virtual HBA's on top of a Xsigo vHBA? Thanks again you guys, your elaborate and quick responses are impressive. Ilja Thank you all for your responses.
Very helpfull and enlightning. But one question goes unanswered.
Maybe I wasn’t clear enough (likely).

vHBA and NPIV make sense to me. Now, I imagine there are restrictions as to how flexible NPIV exposure to one of the connected SAN fabrics is.
I’ll try to clarify my question by using some example ports.

If I were to connect IB port 1 to a host, and created 4 vHBA’s. I obviously would get 4 NPIV ports ready to be exposed to connected FC ports. Can these 4 NPIV ports be exposed to let’s say 4 different FC ports and thereby 4 different fabrics? Or is there a fixed technical boundary were all NPIV ports on an IB port have to be assigned to the same FC port?

Sorry, another question just popped up? Might be a stupid one, but I’ll post it anyway.
The vHBA appears like any regular HBA to the OS, like for example VMware. Within VMware I can also create virtual HBA. Knowing that it would make no sense for real world applications, but would it technically be possible to stack the VMware Virtual HBA’s on top of a Xsigo vHBA?

Thanks again you guys, your elaborate and quick responses are impressive.

Ilja

]]>