Network and Workstation slow since starting to use 2 application servers

[csmithlg]

I am posting this again. I had previously poted on 4.0

We have just put 2 application servers that were on physical server on our network on 2 seperate ESX servers. These were not P to V, they were fresh installs form the OS to the applications. Both attach to a database server also on new build on our 3rd ESX server. We are running everything over it's own GIG switch with all servers and the SAN connections of it. Everything shows at a 1 gig connection, and we checked the MAC addresses as well and all is clear with them. We noticed is since going to new servers is that even workstations not using these programs have shown a decrease in responsiveness.

One program use IIS and the second use IIS along with a local install. Both programs interact with the same database, and on the previous system we did not experiance this. All the resources on the ESX servers show no over load and the same on the Virtual Servers.

Has anyone had this experiance or have a possible solution /suggestion?

3 ESX server (SUNFIRE X4150) 16 gigs of RAM Quad 3.33 MHz

NetApp San with 2 nodes each with 1 TB of storage.

Thanks

[Lightbulb]

Ok let me see if I have this straight.

You had 3 physical servers 2 x App 1x Database. i Take it they were on thesame network and all using localy storage of one kind or another.

You have converted all system to victuals each VM running on separate ESX host ( I assume you are using NFS to store the VMs).

Now something is impacting performance of your IIS based app.

ideas/thoughts/questions:

Is you NFS storage on separate physical network. If not is it on separate VLAN at least?

What kind of Database server are you using? how is it's storage provisioned in relation to previous system.

Are the windows IIS VMs up to the same path/*.NET level as previous physical systems?

Is the DB VM at same patch level as previous physical system (OS/application patches)

Is there a time period where performance is degraded or is performance degraded overall when running app.

Can you rollback to using physical DB system and see if issue lessens or goes away (In my experience the DB system is often the cause of issues with multiple IIS web apps)

[pizmooz]

I would put the SAN portion of the network on it's own switch. It could be the the storage transactions are bogging down the network. As mentioned seperating the storage network via vlans would help, but I think isolating it with it's own physical switch would be best.

[csmithlg]

Is you NFS storage on separate physical network. If not is it on separate VLAN at least?

We are moving it to a second VLAN, and once we have that setup, we are going to move the iSCSI and VMotion to 2 seperate networks. It was not setup on our switch correctly to start.

What kind of Database server are you using? how is it's storage provisioned in relation to previous system.

We are on 64bit 2003 and running SQL 2005. This original was 32 bit 2003 and SQL2000. We did a test environment first to make sure all the SQL data worked with the 2 application servers.

The differance is we had 1 server doing all the work and now we have 3 ( Application server 1, Application server 2, Database server with OS drive and data storage) Each is setup on a different LUN on the SAN.

Are the windows IIS VMs up to the same path/*.NET level as previous physical systems?

Yes

Is the DB VM at same patch level as previous physical system (OS/application patches)

In fact it is a cleaner install.

Is there a time period where performance is degraded or is performance degraded overall when running app.

Prior to going full production, it ran fast. It was after 24 hours it started degrade. No question now it is the need to have them on a differnt VLAN & Network

Can you rollback to using physical DB system and see if issue lessens or goes away (In my experience the DB system is often the cause of issues with multiple IIS web app's)

Yes we could, it would involve several hours of down time. Had we not had a power failure Sunday morning we had that in mind to test. Now keeping in mind there would be a limit to our ability to test without full staff.

Right now we have reconfigured a switch to put in place with the 2 VLANS with no connection to the production LAN to put as I mentioned the iSCSI and vMotion on. That will be the first step.

Thanks you have brought up is very helpful and informative

[csmithlg]

If you see my earlier post, that is the plan. Once we make this change targeted for Wednesday night I will post the results.

thanks