Using Cisco 3750 for iSCSI Switch

[femialpha]

Does any one have 2 3750's configured in a stack for iscsi traffic? I'm attempting to set one up but i can't ping any node connected to the member switch. Any ideas?

[bggb29]

If you remove the stackwise cable can you ping anything.

Are both switches seen when you do a sh running

[thechicco]

I've got a stack of 4 3750G's.

Not able to get to the config at the moment but PM me your sh conf. I'll try and see if I can point you in the right direction.

[femialpha]

I found the problem. The second switch worked fine as a standalone but when i added it to the stack the vlan i assigned to the ports (vlan4) reverted back to default vlan1 which was disabled. I have no clue why that happened so i just reassigned all ports to vlan4 and i'm back in business.

[Dus10]

When a switch becomes a member of stack, it must become subserviant. The stack operates as a unit... having separate configs wouldn't work out so well. It basically becomes a blank slate.

[cent20]

Hello femialpha

do you still ahve the configuration for the 2 stacked 3750 for the isci?

Thanks

[azn2kew]

Would it be better if you architect with two different stacks so that they dont' dependent on when you do firmware update? If you stacked everything up, you might have to power down those stacked together. Any thoughts how to architect these 3750 for redundancies especially iSCSI networks.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[cent20]

Yes, using 2 separate redundant stacks would be the best option. I think the best way to make the stacks redundant is to buit a a port channel trunk bettween the 2 staks, using 1 or 2 ports from each switch in the stack.

But am still not sure about how to confifure the 3750 for iscsi use.

[azn2kew]

That's what I've designed for my iSCSI solution to have two seperate stacks for redundancies but some one may have better options or ideas which I'm always looking forward to learn more.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[formulator]

What model 3750 are you talking about? There are 3750 switches which just have 10/100 ports and then the 3750G which is 10/100/1000 access ports. You def want a 3750G for iSCSI.

For config you would want "system mtu jumbo 9000" which is a global config command and of course put iSCSI traffic on a seperate vlan.

EDIT: I think you might have to reload the switch for the jumbo mtu to take.

[PGRSteve]

Yes on a 3750G when you enable jumbo frames it is a global command and the switch has to be rebooted before jumbo frames are enabled.

[cypherx]

We have a 3750g for our iSCSI, but we do not have jumbo frames turned on. I was told that every single device in the network would require Jumbo Frame support. I didn't feel like going to every single PC in the organization and setting them to Jumbo frames. Not to mention some equipment that may not support it. If a big 9000k packet comes through then the switch has to break it up in 1500k chunks, causing high cpu usage on the switch, fragments and a measureable performance degradation. Now in a greenfield situation with all new equipment I'd say go for it.

[PGRSteve]

you do not have to enable every single device on the network for jumbo frames. If you are implementing an iSCSI solution you would want to use Jumbo frames. you also only need to enable Jumbo frames throughout the iSCSI infrastructure. Meaning any device that will be using Jumbo frames (i.e. Server PNIC's) and any network device (i.e. switch ports and layer 2 and 3 interfaces) that the traffic may flow through. Our jumbo frame implementation is on a secure network that can only be accessed by the servers and switching/ routing gear that reside on it therefore I do not need to enable Jumbo frames anywhere except within this network. i would be suprised if you requirement was to enable every PC for Jumbo frames.

[cypherx]

Thats what I was always told. Because at some point the traffic coming out of our 3750 is going to come out of that switch and talk on the lan. How do you have your clients talk to the servers? Are you putting a router in between your 3750 and the rest of your lan?

We have a 3750 with 3 VLAN's. Ok 1 is the default VLAN for server LAN traffic, VLAN10 is the DMZ and VLAN20 is the SAN. Of course our EMC gear and appropriate NIC's on our vmware hosts are attached to VLAN20, but I don't think you can enable Jumbo Frames on only VLAN20, as it's a global configuration parameter. The default VLAN has 2 links LACP'd to the rest of our flat lan (Linksys/Cisco business class switches). So if 9000k frames come out of the Cisco and into the Linksys, then that Linksys is going to have to break up if a client PC logs on and accesses a service on one of those virtual machines.

I'd be very interested to hear how you could get this to work properly. I'm just going by what our vendors have explained to me, as well as numerous online threads regarding Jumbo Frames.

[formulator]

Only the host/storage ports that are generating iSCSI traffic should be configed for 9000 byte frames. Turning on jumbo frames on the switch just means the switch will be able to process large frames. Once you have that on you just config your iSCSI target for 9000 byte frames and the iSCSI initiators for 9000 byte frames that the switch is now capable of handling. Why would a jumbo frame goto your linksys device? The jumbo frames should stay non-routed on vlan20.

[cypherx]

I guess they didn't want me messing with Jumbo frames on servers if the clients didn't support it. Example would be a File/Print server. Users downloading files off of the file server would have their traffic fragmented if the server was sending out 9000k packets but the client nic could only interpret 1500k at a time. Or an e-mail server, or other application server etc...

I see your point about only taking jumbo frames on the SAN, there would be no issue as long as all the devices that talk to each other have it turned on.

Where in VMWare ESX4 do you set the frame size?

[formulator]

You're missing something here. A client, for a file/print server speaks CIFS to the file/print server to send/receive it's data. The file/print server speaks iSCSI in 9000 byte packets to the target to read/write data from it's storage, it still serves the data back to the client using CIFS and 1500 byte packets. Think of the file/print server as a "proxy" between the client and iSCSI storage.

[cypherx]

Oh yeah I get that now. I wasn't really clear either. I was thinking of doing Jumbo Frames even prior to our whole new Virtual Infrastructure. Prior to having a SAN, etc... We had 20 some rack mount servers all doing their own thing, a flat lan, etc... But now with the SAN I see what your saying and it's def. something for us to consider. I see on our EMC gear where to change the MTU, but now where do I specify the MTU on the vmnic's on each ESX 4.0 host?

[PGRSteve]

i do not think the info is being interpreted properly. if you only enable vlan 20 for Jumbo frames and all interfaces within that vlan including the storage/ targets and PNIC's than only these devices will use jumbo frames. all other will be fragmented if they enter or leave this network via a layer 3 interface.