Question

I am having a problem printing out the correct number of records for a given file. My test script simply does a perl dbi connection to a mysql database and given a list of tables, extracts (1) record per table.

For every table I list, I also want to print that (1) record out to its own file. For example if I have a list of 100 tables, I should expect 100 uniques files with (1) record each.

So far, I am able to generate the 100 files, but there is more than (1) record. There are up to 280 records in the file. Ironically, I am generating a unique key for each record and the keys are unique.

If I print out the $data to a single file (outside the foreach loop), I get the expected results, but in one single file. So one file with 100 records for example, but I want to create a file for each.

I seem to have a problem opening up a file handle and outputting this correctly? or something else is wrong with my code.

Can someone show me how to set this up properly? Show me some best practices for achieving this?
Thank you.

Here is my test code:

# Get list of table
my @tblist = qx(mysql -u foo-bar -ppassw0rd --database $dbsrc -h $node --port 3306 -ss -e "show tables");

#Create data output
my $data = '';
foreach my $tblist (@tblist)
{
   chomp $tblist;

   #Testing to create file
   my $out_file = "/home/$node-$tblist.$dt.dat";
   open (my $out_fh, '>', $out_file) or die "cannot create $out_file: $!";

   my $dbh = DBI->connect("DBI:mysql:database=$dbsrc;host=$node;port=3306",'foo-bar','passw0rd');
   my $sth = $dbh->prepare("SELECT UUID(), '$node', ab, cd, ef, gh, hi FROM $tblist limit 1");
   $sth->execute();
         while (my($id, $nd,$ab,$cd,$ef,$gh,$hi) = $sth->fetchrow_array() ) {
         $data = $data. "__pk__^A$id^E1^A$nd^E2^A$ab^E3^A$cd^E4^A$ef^E5^A$gh^E6^A$hi^E7^D";
         }
         $sth->finish;
         $dbh->disconnect;

   #Testing to create file
   print $out_fh $data;
   close $out_fh or die "Failed to close file: $!";
};
#print $data; #Here if I uncomment and output to a single file, I can see the correct number of record, but its in (1) file

Answer 1

You need to clear $data on each $tblist loop iteration (outer loop).

In this line: $data = $data. "__pk__^A$id^E1^A$... you are appending the data from new table each iteration on TOP of the old data, and it gets preserved in your code between different tables since the $data variable is scoped OUTSIDE the outer loop and its value never gets reset inside of it.

The simplest solution is to declare $data inside the outer ($tblist) loop:

foreach my $tblist (@tblist) {
    my $data = '';

You could keep declaring it before the outer loop and simply assign it "" value at the start of the loop, but there’s no point – there is usually no legitimate reason to know the value of $data in a loop like this after a loop finishes so there’s no need for it to be in the scope bigger than the loop block.