I've just come back from a NetApp training course; good course and recommended for anyone who wants to pick up some storage fundamentals, it covers all the NetApp bases and by the end of it, you should be fairly confident to do pretty much all the day-to-day routine tasks that you might be asked to do as an administrator of a NetApp array.
It does not cover SAN in any detail and the FC coverage is limited to this is how you present a LUN as a fibre channel device but this led on to some interesting conversations around the complexity of FC vs iSCSI.
I, for sometime, have been saying that the complexity of FC is over-stated and actually it is not really any harder than iSCSI. This often leads to looks of disbelief and complete disagreement, it is almost as if I am spouting heresy. The iSCSI camp think I am mad and the FC camp seem to think that I'm diminishing them.
But, if you take OnTap; there is really very little difference in how you present a iSCSI LUN compared to how you present an FC LUN. It is certainly no harder to do FC from an array management point of view.
And then we go to the host; lets take Windows for example. Now this is where I think a lot of the perception of the simplicity of iSCSI comes in; Microsoft have implemented a very nice software initiator, it's there and it's standard. A bit of pointing and clicking and you are there.
Unfortunately, traditional FC cannot be implemented completely in software and needs FC HBAs, hence we need to install additional drivers and software to make it work; these are not part of the standard Windows build and suddenly it all becomes 'complex'.
If we go to Unix, we end up mucking about configuration files in general for both iSCSI and FC; so really it's not any harder to do FC than iSCSI.
So if it's not hard to do at the host level and if it's not hard to do at the array; where is it hard? And this is where I think it becomes more interesting; it's the network! Is a large Data Centre IP network harder to set-up than a large Data Centre FC network?
Arguably, the FC network is easier but it is different. In the FC network you have a lot less to worry about; you run less protocols, services, it's non routeable, the security model is simpler, there is less potential for different workloads to clash, you do not have address space management to worry, you do not have name services to worry about and if I were a Network Admin, I would argue that it's them who are being diminished by this constant claim that iSCSI is easier.
To do either iSCSI or FC properly is probably equally hard. If you just want to bung a block-level array in and do not care about segregation of traffic, quality of service, don't care whether your IP back-up traffic and IP storage traffic contend and make your back-ups over-run. If you know you've got enough headroom on your existing IP network to carry your block traffic, go ahead with iSCSI; it's easier because you've already got the infrastructure in place.
But if you want put in place a dedicated storage network; the choice is not as clear-cut as people would like to make out. Even when you start looking at cost; yes GbE is cheaper than FC but FC is generally running at 4 gig now and is faster. FC ports are in my experience are significantly cheaper than 10GbE ports. So if you need the throughput, then FC might well be cheaper than iSCSI.
There might be a small saving in the number of FTEs you require as you could have a single Network team but I believe that FC is actually so simple that if you get over the politics, you could have a single Network team which manages both FC and IP. This is purely politics and a turf-war!
If you are a small shop and you have just a few administrators who do everything, iSCSI might also make sense but don't believe FC is hard; administering a small SAN with a couple of switches might not add a huge amount of additional overhead.
Neither iSCSI or FC are wrong answers but make sure that when you have got to the answer, you show your working. And when a vendor tells you what the answer is, ask them to show your their working and challenge it.
Of course, if I was building a green-field data-centre and could simply start again, I'd probably look at putting in Data Centre Ethernet which would give me the option of FCoE. I would have a single Network team from the get go.
It'll be interesting to see if Microsoft bundle a software initiator for FCoE into Windows at some point; then I think we'll see perceptions of complexity change again.
Yes, I've completely ignored NAS but actually many of the same arguments apply.
I'd welcome some thoughts; that's if anyone is still reading and not exploded in apoplectic rage at the heretic!
I don’t think that there’s anything heretical about the advantages of simplicity — regardless of protocol.
And I don’t think there’s anything heretical about convergence of protocols on a single wire, as long as you get behavioral and management model compatibility.
Where I beg to differ with some is when a given solution is positioned as “all you’ll ever need”, as the world is a complex place.
I’m with you that in a pure technical sense FC isn’t really more difficult, but that’s once you get to the other side of the learning curve. IP is familiar to most everyone in IT. The knowledge needed to implement iSCSI is incremental in nature, and a small increment at that. The network team already knows the protocols for building out a robust iSCSI network – no worrying about domain ID’s, ISL’s, FLOGI’s, pWWN, nWWN, zones, etc. And lets not even get into SAN extension… I also think that the margin or error is smaller on FC than iSCSI, and after an error FC is less forgiving.
Even if FC is already in place I’d say that it takes almost twice as long to setup an FC host than iSCSI (assuming standard NIC’s). Need to dig through an interoperability matrix and download the correct HBA firmware and NVRAM setting, then build the zones. I also find that getting change requests processed takes longer with FC in many organizations.
Ryan, good points! The conversations on the course came up some interesting observations; anecdotal evidence suggests that IP networks seem to suffer more problems than FC networks; especially due to changes. Perhaps IP networks could do with more rigorous application of change controls.
In the organisation I work for, we see very few issues with changes in the FC storage network and we actually work to the same SLAs and OLAs as the network boys to turn around requests.
Also, I find significant holes in the understanding of IP; a lot of people who claim familiarity with IP, actually only have a very vague idea. As soon as you drift outside of basic IP addressing, it all goes to pot. But because their home desktop speaks IP, they believe that they understand IP and hence are not scared of iSCSI because it feels vaguely familiar.
And as I say, it’ll be interesting to see if Microsoft puts a software FCoE initiator into Windows at some point and whether that breeds a familiarity with FC protocols.
I’ve made it very clear that I don’t believe in one true answers…I don’t sit in either camp especially but I do think that some of the much vaunted simplicity of iSCSI is overstated.
I’d like to give Ryan Benigno a pat on the back. He’s dead on!
The problem of FC is that FC is unfamiliar, plain and simple. Nothing will ever change that (except some wild fantasy world in which FC is the predominant storage connector in consumer computers) so we have to deal with it. This means redirecting new users to the more familiar waters of iSCSI/IP/Ethernet and the more-friendly Windows iSCSI initiator.
Note that Windows Server 2008 R2/Windows 7 has a MUCH more friendly initiator (see the fall WinHEC presentation by Suzanne Morgan) called iSCSI Quick Connect. Apparently my incessant “suggestions” were taken up by the product team!
If we stick with things simply because they are familiar, things would never change 😉 All I ask, is that people think about what they are doing.
I speak to people all too often who accept default answers without thinking why. And I wonder how you can make the Windows initiator any more friendly, it is already insanely easy.
I pretty much agree with your post. For some reason admins are afraid of the orange, now sometimes gray or pink, wire. With the complete separation of FC and IP traffic it makes troubleshooting easier.
Throw Netapp into the mix and there is no longer a need to zone.