Question

I have a Perl CGI script that isn’t working and I don’t know how to start narrowing down the problem. What can I do?

---

Note: I’m adding the question because I really want to add my very lengthy answer to Stack Overflow. I keep externally linking to it in other answers and it deserves to be here. Don’t be shy about editing my answer if you have something to add.

Answer 1

This answer is intended as a general framework for working through
problems with Perl CGI scripts and originally appeared on Perlmonks as Troubleshooting Perl CGI Scripts. It is not a complete guide to every
problem that you may encounter, nor a tutorial on bug squashing. It
is just the culmination of my experience debugging CGI scripts for twenty (plus!) years. This page seems to have had many different homes, and I seem
to forget it exists, so I’m adding it to the StackOverflow. You
can send any comments or suggestions to me at
[e-mail]. It’s also community wiki, but don’t go too nuts. 🙂

---

Are you using Perl’s built in features to help you find problems?

Turn on warnings to let Perl warn you about questionable parts of your code. You can do this from the command line with the -w switch so you don’t have to change any code or add a pragma to every file:

 % perl -w program.pl

However, you should force yourself to always clear up questionable code by adding the warnings pragma to all of your files:

 use warnings;

If you need more information than the short warning message, use the diagnostics pragma to get more information, or look in the perldiag documentation:

 use diagnostics;

Did you output a valid CGI header first?

The server is expecting the first output from a CGI script to be the CGI header. Typically that might be as simple as print "Content-type: text/plain\n\n"; or with CGI.pm and its derivatives, print header(). Some servers are sensitive to error output (on STDERR) showing up before standard output (on STDOUT).

Try sending errors to the browser

Add this line

 use CGI::Carp 'fatalsToBrowser';

to your script. This also sends compilation errors to the browser window. Be sure to remove this before moving to a production environment, as the extra information can be a security risk.

What did the error log say?

Servers keep error logs (or they should, at least).
Error output from the server and from your script should
show up there. Find the error log and see what it says.
There isn’t a standard place for log files. Look in the
server configuration for their location, or ask the server
admin. You can also use tools such as CGI::Carp
to keep your own log files.

What are the script’s permissions?

If you see errors like "Permission denied" or "Method not
implemented", it probably means that your script is not
readable and executable by the web server user. On flavors
of Unix, changing the mode to 755 is recommended:
chmod 755 filename. Never set a mode to 777!

Are you using use strict?

Remember that Perl automatically creates variables when
you first use them. This is a feature, but sometimes can
cause bugs if you mistype a variable name. The pragma
use strict will help you find those sorts of
errors. It’s annoying until you get used to it, but your
programming will improve significantly after awhile and
you will be free to make different mistakes.

Does the script compile?

You can check for compilation errors by using the -c
switch. Concentrate on the first errors reported. Rinse,
repeat. If you are getting really strange errors, check to
ensure that your script has the right line endings. If you
FTP in binary mode, checkout from CVS, or something else that
does not handle line end translation, the web server may see
your script as one big line. Transfer Perl scripts in ASCII
mode.

Is the script complaining about insecure dependencies?

If your script complains about insecure dependencies, you
are probably using the -T switch to turn on taint mode, which is
a good thing since it keeps you have passing unchecked data to the shell. If
it is complaining it is doing its job to help us write more secure scripts. Any
data originating from outside of the program (i.e. the environment)
is considered tainted. Environment variables such as PATH and
LD_LIBRARY_PATH
are particularly troublesome. You have to set these to a safe value
or unset them completely, as I recommend. You should be using absolute
paths anyway. If taint checking complains about something else,
make sure that you have untainted the data. See perlsec
man page for details.

What happens when you run it from the command line?

Does the script output what you expect when run from the
command line? Is the header output first, followed by a
blank line? Remember that STDERR may be merged with STDOUT
if you are on a terminal (e.g. an interactive session), and
due to buffering may show up in a jumbled order. Turn on
Perl’s autoflush feature by setting $| to a
true value. Typically you might see $|++; in
CGI programs. Once set, every print and write will
immediately go to the output rather than being buffered.
You have to set this for each filehandle. Use select to
change the default filehandle, like so:

$|++;                            #sets $| for STDOUT
$old_handle = select( STDERR );  #change to STDERR
$|++;                            #sets $| for STDERR
select( $old_handle );           #change back to STDOUT

Either way, the first thing output should be the CGI header
followed by a blank line.

What happens when you run it from the command line with a CGI-like environment?

The web server environment is usually a lot more limited
than your command line environment, and has extra
information about the request. If your script runs fine
from the command line, you might try simulating a web server
environment. If the problem appears, you have an
environment problem.

Unset or remove these variables

• PATH
• LD_LIBRARY_PATH
• all ORACLE_* variables

Set these variables

• REQUEST_METHOD (set to GET, HEAD, or POST as appropriate)
• SERVER_PORT (set to 80, usually)
• REMOTE_USER (if you are doing protected access stuff)

Recent versions of CGI.pm ( > 2.75 ) require the -debug flag to
get the old (useful) behavior, so you might have to add it to
your CGI.pm imports.

use CGI qw(-debug)

Are you using die() or warn?

Those functions print to STDERR unless you have redefined
them. They don’t output a CGI header, either. You can get
the same functionality with packages such as CGI::Carp

What happens after you clear the browser cache?

If you think your script is doing the right thing, and
when you perform the request manually you get the right
output, the browser might be the culprit. Clear the cache
and set the cache size to zero while testing. Remember that
some browsers are really stupid and won’t actually reload
new content even though you tell it to do so. This is
especially prevalent in cases where the URL path is the
same, but the content changes (e.g. dynamic images).

Is the script where you think it is?

The file system path to a script is not necessarily
directly related to the URL path to the script. Make sure
you have the right directory, even if you have to write a
short test script to test this. Furthermore, are you sure
that you are modifying the correct file? If you don’t see
any effect with your changes, you might be modifying a
different file, or uploading a file to the wrong place.
(This is, by the way, my most frequent cause of such trouble
😉

Are you using CGI.pm, or a derivative of it?

If your problem is related to parsing the CGI input and you
aren’t using a widely tested module like CGI.pm, CGI::Request,
CGI::Simple or CGI::Lite, use the module and get on with life.
CGI.pm has a cgi-lib.pl compatibility mode which can help you solve input
problems due to older CGI parser implementations.

Did you use absolute paths?

If you are running external commands with
system, back ticks, or other IPC facilities,
you should use an absolute path to the external program.
Not only do you know exactly what you are running, but you
avoid some security problems as well. If you are opening
files for either reading or writing, use an absolute path.
The CGI script may have a different idea about the current
directory than you do. Alternatively, you can do an
explicit chdir() to put you in the right place.

Did you check your return values?

Most Perl functions will tell you if they worked or not
and will set $! on failure. Did you check the
return value and examine $! for error messages? Did you check
$@ if you were using eval?

Which version of Perl are you using?

The latest stable version of Perl is 5.28 (or not, depending on when this was last edited). Are you using an older version? Different versions of Perl may have different ideas of warnings.

Which web server are you using?

Different servers may act differently in the same
situation. The same server product may act differently with
different configurations. Include as much of this
information as you can in any request for help.

Did you check the server documentation?

Serious CGI programmers should know as much about the
server as possible – including not only the server features
and behavior, but also the local configuration. The
documentation for your server might not be available to you
if you are using a commercial product. Otherwise, the
documentation should be on your server. If it isn’t, look
for it on the web.

Did you search the archives of comp.infosystems.www.authoring.cgi?

This use to be useful but all the good posters have either died or wandered off.

It’s likely that someone has had your problem before,
and that someone (possibly me) has answered it in this
newsgroup. Although this newsgroup has passed its heyday, the collected wisdom from the past can sometimes be useful.

Can you reproduce the problem with a short test script?

In large systems, it may be difficult to track down a bug
since so many things are happening. Try to reproduce the problem
behavior with the shortest possible script. Knowing the problem
is most of the fix. This may be certainly time-consuming, but you
haven’t found the problem yet and you’re running out of options. 🙂

Did you decide to go see a movie?

Seriously. Sometimes we can get so wrapped up in the problem that we
develop "perceptual narrowing" (tunnel vision). Taking a break,
getting a cup of coffee, or blasting some bad guys in [Duke Nukem,Quake,Doom,Halo,COD] might give you
the fresh perspective that you need to re-approach the problem.

Have you vocalized the problem?

Seriously again. Sometimes explaining the problem aloud
leads us to our own answers. Talk to the penguin (plush toy) because
your co-workers aren’t listening. If you are interested in this
as a serious debugging tool (and I do recommend it if you haven’t
found the problem by now), you might also like to read The Psychology
of Computer Programming.