Question

I just introduced threads to a Perl program, where one of its modules was using Memoize.
I’m getting this error message:

Thread 1 terminated abnormally: Anonymous function called in forbidden scalar context; faulting

The error occurs if I have both threads and Memoize, but will disappear if I take away one of these elements. But the problem isn’t because Memoize isn’t thread-safe – in my code, all the memoization happens within the same thread.

Is this a bug with Memoize? Is there a way I can work around this? Otherwise I’m going to get rid of Memoize.

Here’s some sample code to isolate the problem:

use strict;
use warnings;
use threads;
use Thread::Semaphore;
use Memoize;

my $semaphore = Thread::Semaphore->new;

memoize('foo');
sub foo {
    return shift;
}

sub invoke_foo {
    $semaphore->down; # ensure memoization is thread-safe
    my $result = foo(@_);
    $semaphore->up;

    return $result;
}

my @threads;
foreach (1 .. 5) {
    my $t = threads->create( sub { invoke_foo($_) });
    push @threads, $t;
}
$_->join foreach @threads;

Answer 1

Memoize stores the caches for every memoized function in one hash (instead of using a closure). It uses the address of the function as the index into that hash.

The problem is that the address of the function changes when it’s cloned into a new thread. (Add print(\&foo, "\n"); in invoke_foo.). It’s a bug in Memoize.

Workaround: Load the memoised module from within the threads. the following simulates (the relevant aspects of) that:

use strict;
use warnings;
use threads;
use Memoize;

sub foo {
    return shift;
}

sub invoke_foo {
    return foo(@_);
}

my @threads;
foreach (1 .. 5) {
    my $t = threads->create( sub {
        memoize('foo');
        invoke_foo($_);
    });
    push @threads, $t;
}
$_->join foreach @threads;

By the way, each thread has its own cache. that could also be considered a bug.