Question

I have part of a build process that creates a hideously long paths in Windows. It’s not my fault. It’s several directories deep, and none of the directory names are abnormally long; they’re just long and numerous enough to make it over MAX_PATH (260 chars). I’m not using anything other than ASCII in these names.

The big problem is that the blow-up happens deep in the guts of Module::Build during the dist target, although I figure the build system doesn’t matter because they’d make the same directories.

Creating one of these overly-long directories with File::Path fails:

 use File::Path qw( make_path );

 make_path( 'C:\\.....' ); # fails if path is over 260 chars

Similarly, constructing each directory level by hand fails once the absolute path would go over MAX_PATH.

This isn’t new, isn’t Perl’s fault, and Microsoft documents it in Naming Files, Paths, and Namespaces. Their fix suggests adding the \\?\ in front of any path to access the Unicode filename API. However, that doesn’t seem to be the full fix for a Perl script because it still fails:

 use File::Path qw( make_path );

 make_path( '\\\\?\\C:\\.....' );  # still fails if path is over MAX_PATH, works otherwise

This might be because make_path pulls apart its argument and then goes through the directories one level at a time, so \\?\ only applies to the top-level, which is within MAX_PATH.

I dug up a bug report to ActiveState that suggests there’s something else I need to fix up to get to the Unicode filenames, and Jan Dubois gives a bit more details in Re: “long” filenames on Windows 2K/XP, although I’m not sure it applies (and is extremely old). perlrun mentions that this use to be the job of the -C switch, but apparently that part was abandoned. The perl RT queue has a more recent bug 60888: Win32: support full unicode in filenames (use Wide-system calls).

Miyagawa notes some Unicode filename issues and Win32API::File without specifically mentioning long paths. However, the Win32API::File CPAN Forum entry seems to indicate only fear, which leads to anger, which leads to hate, and so on. There’s an example in the Perlmonks post How to stat a file with a Unicode (UTF16-LE) filename in Windows?. It seems the Win32::CreateDirectory is the answer, and I’ll try that the next time I get next to a Windows machine.

Then, supposing I can create the long path path. Now I have to teach Module::Build, and maybe other things, to handle it. That might be immediately easy with monkeypatches if Win32::GetANSIPathName() does what it says on the tin.

Answer 1

Windows has two separate system call for each function that needs to deal with strings, an "A" call using the ANSI aka Active Code Page as the encoding (e.g. cp1252) and a "W" call using UTF-16le. Perl uses "A" calls, while \\?\ only works with "W" calls.

You can use Win32::API to access the "W" calls as shown in the script below, but Win32::LongPath not only uses the "W" calls, but automatically adds \\?\!

Example of using Win32::API to call CreateDirectoryW to use a long path (\\?\-prefixed path):

#!/usr/bin/perl

use strict;
use warnings;

use Carp;
use Encode qw( encode );
use Symbol;

use Win32;

use Win32API::File qw(
    CreateFileW OsFHandleOpen
    FILE_GENERIC_READ FILE_GENERIC_WRITE
    OPEN_EXISTING CREATE_ALWAYS FILE_SHARE_READ
);

use Win32::API;
use File::Spec::Functions qw(catfile);

Win32::API->Import(
    Kernel32 => qq{BOOL CreateDirectoryW(LPWSTR lpPathNameW, VOID *p)}
);

my %modes = (
    '<' => {
        access => FILE_GENERIC_READ,
        create => OPEN_EXISTING,
        mode   => 'r',
    },
    '>' => {
        access => FILE_GENERIC_WRITE,
        create => CREATE_ALWAYS,
        mode   => 'w',
    },
    # and the rest ...
);

use ex::override open => sub(*;$@) {
    $_[0] = gensym;

    my %mode = %{ $modes{$_[1]} };

    my $os_fh = CreateFileW(
        encode('UCS-2le', "$_[2]\0"),
        $mode{access},
        FILE_SHARE_READ,
        [],
        $mode{create},
        0,
        [],
    ) or do {$! = $^E; return };

    OsFHandleOpen($_[0], $os_fh, $mode{mode}) or return;
    return 1;
};

my $path = '\\\\?\\' . Win32::GetLongPathName($ENV{TEMP});
my @comps = ('0123456789') x 30;

my $dir = mk_long_dir($path, \@comps);
my $file = 'test.txt';
my $str = "This is a test\n";

write_test_file($dir, $file, $str);

$str eq read_test_file($dir, $file) or die "Read failure\n";

sub write_test_file {
    my ($dir, $file, $str) = @_,

    my $path = catfile $dir, $file;

    open my $fh, '>', $path
        or croak "Cannot open '$path':$!";

    print $fh $str or die "Cannot print: $!";
    close $fh or die "Cannot close: $!";
    return;
}

sub read_test_file {
    my ($dir, $file) = @_,

    my $path = catfile $dir, $file;

    open my $fh, '<', $path
        or croak "Cannot open '$path': $!";

    my $contents = do { local $/; <$fh> };
    close $fh or die "Cannot close: $!";
    return $contents;
}

sub mk_long_dir {
    my ($path, $comps) = @_;

    for my $comp ( @$comps ) {
        $path = catfile $path, $comp;
        my $ucs_path = encode('UCS-2le', "$path\0");
        CreateDirectoryW($ucs_path, undef)
            or croak "Failed to create directory: '$path': $^E";
    }
    return $path;
}