Question

When Module::Starter initializes a project, it creates a test called manifest.t.

#!perl -T

use strict;
use warnings;
use Test::More;

unless ( $ENV{RELEASE_TESTING} ) {
    plan( skip_all => "Author tests not required for installation" );
}

eval "use Test::CheckManifest 0.9";
plan skip_all => "Test::CheckManifest 0.9 required" if $@;
ok_manifest();

When you run tests with Build test, here’s part of the output:

t\00-load.t ....... ok
t\boilerplate.t ... ok
t\manifest.t ...... skipped: Author tests not required for installation

I understand the outcome in a narrow sense ($ENV{RELEASE_TESTING} is not set, so the tests are skipped), but I don’t fully grasp the big picture. What’s the intended development process? I assume it’s a good idea to run tests to confirm that my module’s manifest is accurate. Should I be setting that environment variable? If so, at what point during the development process?

Answer 1

Many module distributions have tests that check not whether the code works, but whether the distribution is in a suitable state for releasing. Things like the MANIFEST being up to date, whether all functions have been documented in POD, etc.

In order to save time, these tests may be written to skip themselves unless the RELEASE_TESTING environment variable is set. This is an informal standard. That way, these tests don’t get run when people install the module, nor do they run when the author is just checking to see if a code change broke anything.

You should run RELEASE_TESTING=1 make test (or the Build equivalent) before releasing your dist. If you use Dist::Zilla (which I highly recommend), you can run release tests with dzil test --release. That flag is also set automatically by the TestRelease plugin, which you should definitely use if you use dzil.

Other environment variables commonly used to control testing are AUTOMATED_TESTING and AUTHOR_TESTING. AUTOMATED_TESTING is set by CPAN testers running automated smoke tests.