I have a problem running a printserver on top of VMware ESX.
The machine is running with a very high cpu-load.
First I want to describe the installation:
Hostname: cobu0035
OS: SLES 10 SP1
Kernel: Linux version 2.6.16.54-0.2.10-smp
VM-Tools: Version 3.5.0, build-123630
CPU's: Log: HOSTINFO: Seeing Intel CPU, numCoresPerCPU 4 numThreadsPerCore 1.
Log: HOSTINFO: numPhysCPUs is 0, bumping to 1.
Log: HOSTINFO: This machine has 1 physical CPUS, 2 total cores, and 2 logical CPUs.
bootparameter:
root=/dev/system/lvroot resume=/dev/system/lvswap splash=silent showopts clock=notsc
Application:
The machine is running a cups Printserver Version: cups-1.1.23-40.46
And a Samba Fileserver as frontend to host the printerdrivers.
The server is running very well. The machines is hosting 220
printers with approx. 5000-6000 printjobs per day.
Problem:
The machines seems to be very slow, when somebody try to connect
to the samba-shares.
To track down this issue I have done the following steps:
When I do a uptime I got the following output:
cobu0035:/usr/lib/vmware-tools/bin32 # uptime
11:39am up 4 days 18:35, 2 users, load average: 1.75, 1.97, 1.88
The load average seems to be very high!
When I do a top I got this output:
cobu0035:/usr/lib/vmware-tools/bin32 # top
top - 11:41:13 up 4 days, 18:36, 2 users, load average: 1.57, 1.83, 1.83
Tasks: 337 total, 5 running, 332 sleeping, 0 stopped, 0 zombie
Cpu0 : 19.4%us, 23.7%sy, 0.0%ni, 55.6%id, 0.3%wa, 0.0%hi, 1.0%si, 0.0%st
Cpu1 : 41.3%us, 20.1%sy, 0.0%ni, 36.3%id, 0.0%wa, 1.0%hi, 1.3%si, 0.0%st
Mem: 2075592k total, 1563360k used, 512232k free, 95196k buffers
Swap: 1048568k total, 108k used, 1048460k free, 192624k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19137 root 18 0 99108 92m 1568 R 33 4.5 305:04.11 cupsd
9431 BROSE+lo 15 0 44808 9160 4452 S 26 0.4 0:05.16 smbd
10759 BROSE+zi 15 0 49972 10m 6420 S 20 0.5 0:01.59 smbd
3140 root 15 0 55344 10m 8584 S 6 0.5 128:09.84 winbindd
When we look into the two lines with Cpu0 and Cpu1 there is a very high
systmload. This means the application cupsd does a lot of systemcalls.
I followed the process via strace and got a
output like this:
...
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
select(1024, [0 2 3 5 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 24 25 26 27
28 29 30 31 32 33 34 35 36 38 39 42 43 44 45 46 47 48 49 50 51 52 53 54 55
56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 78 79 80 81
82 83 84 85 86 87 88 89 90 91 92 95], 22 24 26 28 35 38, NULL, {1, 0}) = 6
(out 22 24 26 28 35 38, left {1, 0})
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
time(NULL) = 1226659685
....
this is repeating and repeating.
It looks very boring, but this means the software is always asking about the
time!
Do I have any chances to speedup this calles?
I know it is wellknow that VMware often has problem with timesync, so we added the clock=notsc but no change.
Any Ideas?
Hello,
Moved to the Virtual Machine and Guest OS forum.
The select statement is polling the system time to know whether or not the select should expire so something else can happen or the select can be called once again. This is very common with the use of select. Basically that tells me that the print daemon could be waiting to handle a job. Then it generally calls another program to handle the print job. So it is basically always waiting.
The load within the VM is artificial and should only be used as a guage to see if more or less CPU is being used. The real CPU usage for the VM is reported in the VIC. What does this show?
Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll
Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links