Question

I cannot make my Perl script run stable on the server.
Here is the problem.

When the script is accessed more than 5 times a second, the server freezes.
And some time later the server hangs forever.
SSH does not respond and I have to restart the server.

I’m using Apache with mod_perl.

The script is hosted on Virtual Dedicated Server under Ubuntu.
I’m operating it through SSH.
These are the server params
CPU: 400 MHz
RAM: 256 MB

Maximal execution time of the script is 200 milliseconds.

I have monitored server load with the “top” utility.
It does not display any problems, this is the CPU statistics during a load of 5 scripts per second:

Cpu(s): 12.1%us,  0.6%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si, 87.2%st

What options do I have to make the script work without problems?

This is the result of ps aux | fgrep perl at the moment of loading:

ps aux | fgrep perl
www-data  2925  0.3  6.5  45520 17064 ?        R    17:00   0:01 /var/www/perl/loa -k start
www-data  2926  0.2  6.5  45520 17068 ?        R    17:00   0:01 /var/www/perl/loa -k start
www-data  2927  0.4  6.5  45676 17060 ?        R    17:00   0:01 /var/www/perl/loa -k start
www-data  2928  0.3  6.5  45676 17060 ?        R    17:00   0:01 /var/www/perl/loa -k start
www-data  2929  0.2  6.5  45676 17060 ?        R    17:00   0:01 /var/www/perl/loa -k start
www-data  2931  0.4  6.5  45740 17076 ?        R    17:00   0:01 /var/www/perl/loa -k start
root      2968  0.0  0.2   3196   656 pts/0    R+   17:06   0:00 fgrep perl

UPDATE

I have found the bottleneck.
I’ve been using DateTime module many times around the code.
The following DateTime module methods appear to be very slow.

• new()
• now()
• set(…)
• delta_ms(…)

I’m going to substitute them with fast analogs.

Another concern.
mod_perl instance takes a lot of memory.
And I have no idea why.
I have tried to run a simple perl script that does not import any modules.
I run it just after apache restart.
The script takes 37M of memory.
Why does it happen?
Do you know how to force mod_perl do not use the extra memory?

A regular perl script, without mod_perl support, takes 3-5M of memory.

Guys, thank you for so much help, I wasn’t expecting such a wonderful response!

UPDATE 2

I have found one more fact.
I’ve created a simple perl script that just waits for 5 seconds.

#!/usr/bin/perl
use CGI;

my $query= new CGI;
my $content = "5 second delay...\n";

$query->header(
    '-Content-type' => "text/plain",
    '-Content-Length' => length($content)
);

print $content;

sleep(5);

Then I spawn many these scripts at the same time.
Stealth time (st) in top utility jumps up from 0% to 80% and stays high until the scripts are done.

Where does this load come from?

Also, as I’ve already mentioned, each perl instance takes 36M of memory.

Answer 1

Your numbers from top seem to indicate that other processes outside your VM are throttling your CPU, note the last number, 87.2%st, which indicates that about 87% of your CPU time is being allocated by your hypervisor for tasks outside your VM even though your VM has things it would like to run. Whether this is related to your problem or not is hard to say.

Beyond upgrading your server as suggested by unwind, using a persistent process environment as suggested by zoul, it’s possible that your process isn’t CPU bound at all but is instead IO-bound, such as to the network or to your disk access, or memory-bound. It’s hard to say without more details on what your script is actually doing when it’s invoked.

EDIT: Your updated question with info on your memory usage is revealing, as each of your processes wants 45M of ram all to itself, and is sharing 17M more. With just 5 or 6 processes running, you’re exceeding the amount of RAM available. That’s a good amount of memory for a vanilla Perl script to use, what’s it doing with it?