Server virtualization is a big thing these days, so I’m tasked at work to install some of our software on a virtualized server and see what happens. Long story short: a rsync transfer promptly brings the virtualized server to its knees. The virtualization host is a beefy machine with no other load; I don’t think this should be happening. Top shows high load averages, and cpu iowait near 100%. There’s a huge bottleneck somewhere.
I’m more a programmer than a sysadmin, I lack the knowledge on how to go about fixing this outside of random Googling. I suspect I’m not alone in this.
What I’d like to see here is general advice on virtualization, and pointers to good articles and other resources, which I and others could use to educate ourselves.
- What tools (even standard unix tools) can be used to pinpoint bottlenecks?
- What metrics should be followed to ensure things run smoothly?
- What kind of things can be efficiently virtualized?
- What kind of setups are doomed to fail?
I apologize the broadness of the question. I just don’t have the knowledge to ask useful specific questions about this.
Edit: More on my specific problem:
- XAN paravirtualization, 3 x guest CentOS
- All guests on local SCSI disks, there is a fully hardware raid controller
- rsyncd running on 1 guest os, transfer initiated from a remote non virtualized server through 100mbps LAN
Like I said before, I really can’t provide a ton of useful data. I’m not really expecting to get a direct solution to this problem, I’d be happy with pointers on where to start building the skillset required to better understand these kinds of problems.
I was going to put this in a comment, but I think it’s more useful in the open:
Could you add more detail about your setup:
(VMware Server, VMware ESX, MS VirtualServer, MS Hyper-V, something else?)
(Windows, Linux, 32-bit, 64-bit?)
(Local disk, or on a NAS or SAN?)
Performance tuning in any environment is 90% collecting data, and 10% analysis. Virtualized environments have more variables to consider than Physical environments, but more importantly, they have a different response curve than physical environments. Some applications can perform better in a virtualized environment than a physical environment; others will not. You have to understand the requirements of the application as well as constraints of the implementation.
I don’t believe that there are any software-only applications that cannot be successfully deployed on virtual servers, if you pay attention to the details. (Applications that require custom hardware that can’t be successfully virtualized is a different problem.)