Question

That was a long question and I could not get it all in. But, I am trying to do something without MySQL here and use files. Not the preferred method but, I need to figure it out.

1 -I want to open a file of emails that are in a single column.

[e-mail]
[e-mail]
[e-mail]

Could be 100k lines!

2- I then want to strip the first two characters of the email and create folders.
(“e/em” “e/ev” or “e/et” per example *1(naturally if !exists))

3- Create a txt file named the two letters *1(if !exists). (path example = e/em/em.txt e/ev/ev.txt and e/et/et.txt)

4- Append those files with UNIQUE emails that start with the said first two letters.
(so, e/em/em.txt would contain [e-mail], e/ev/ev.txt would contain [e-mail] etc. )

I know it is nuts. But that’s what I need to do. (I am so spoiled by MySQL).

My attempt to do this was so miserable and time consuming… I just had to come here for guidance.

I am happy to install a file handling module if it would help.

*1 If it is beneficial to avoid directory and file checks every time, I would like to run a script that created all the possible folders and populate them with each folders empty files ahead of time. Creating 26 folders (a-z) each containing the 26 possible combinations (/aa /ab /ac) all containing appropriate two letter blank file created.

I need some lessons on how to do all this. Although silly, I still need to know how.

Addition:

The directories and filenames can in fact start with – or _

Still needs tweaking but thanks for the help:

#!/usr/bin/perl -w
use strict;
use warnings;
use CGI;
use CGI ':standard';
print CGI::header();
use File::Basename;
use File::Path qw/make_path/;
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
my $path='/home/xxxx/public_html/some/directory';
my $file='1.txt';
my %EmailAddresses;
  open my $IN, '<', $path.'/'.$file or die $!;
   while (<$IN>) {
   chomp;                
   $_=~ s/\s//g;
   undef $EmailAddresses{$_};
  }
for my $EmailAddress(keys %EmailAddresses) {
 ## need to sanitize substr here for use below
  my $filename= join '/', substr($EmailAddress,0,1), substr($EmailAddress,0,2), substr($EmailAddress,0,2) . '.txt';
  $filename = $path.'/'.$filename;
  my $dir = dirname($filename);
   make_path($dir) unless -d "$dir";
     open (OUT, '>>', $filename) || die $!;
     #need to check for dupes and remove other possible issues!
     print OUT $EmailAddress, "\n";
     close OUT;
  }

Answer 1

I used a hash to get the unique e-mails. You have a lot of redundant steps in your structure, though – I would remove the second level directory, its name is the same as the filename, anyway.

#!/usr/bin/perl
use warnings;
use strict;

my %emails;
open my $IN, '<', '1.txt' or die $!;
while (<$IN>) {
    chomp;
    undef $emails{$_};
}

for my $email (keys %emails) {
    open my $OUT, '>>', join '/', substr($email,0,1), substr($email,0,2),
        substr($email,0,2) . '.txt' or die $!;
    print {$OUT} $email, "\n";
}

First grouping the emails in a hash and then printing file by file is faster:

#!/usr/bin/perl
use warnings;
use strict;

my %emails;
open my $IN, '<', '1.txt' or die $!;
while (<$IN>) {
    chomp;
    undef $emails{substr($_, 0, 1)}{substr($_, 0, 2)}{$_};
}

for my $one (keys %emails) {
    for my $two (keys %{ $emails{$one} }) {
        open my $OUT, '>', join '/', $one, $two, $two . '.txt' or die $!;
        print {$OUT} "$_\n" for keys %{ $emails{$one}{$two} };
    }
}