Question

Hej sharp minds!

I need to load text file into hash. One line is the key, next line is the value. And that repeats several million times. Any suggestions how to do it in the best way?

And how much memory the hash table would need if let’s say key is 15 characters and value is 50 characters?

Thanks

Answer 1

The following code should load the text file into the hash:

my %hash;

while (chomp(my $key = <DATA>)) {
    chomp(my $val = <DATA>);
    $hash{$key} = $val;
}

The memory overhead of the hash entries will depend on the architecture (32 vs. 64 bit) but should be on the order of a couple hundred bytes for the hash itself, and then about 30-60 bytes per key and value, plus the overhead of the key and value data types. You can use Devel::Size to check it yourself. Also read this.

So in your example, on a 64-bit platform, a million entries should cost roughly:

136 for the hash

58 + 15 + 58 + 50 == 181 per key/value pair x 1,000,000

181MB for a million entries of the sizes you specified.