Question

I posted this question here before but there were no responses. I may have done something wrong so, here it is again with some more details.

The files in the directory are named 1.txt, 2.txt, 3.txt etc…. The snippet below enters that directory, opens all the *,txt files reading them, removes the dupes and creates one file with all the unique contents. (names in this case).

$files = glob($dirname."/*.txt"); //matches all text files
    $lines = array();
    foreach($files as $file)
    {
    $lines = array_merge($lines, file($file, FILE_SKIP_EMPTY_LINES | FILE_IGNORE_NEW_LINES));
    }
    $lines = array_unique($lines);
    file_put_contents($dirname."/allofthem.txt", implode("\n", $lines));
    }

The above works great for me! Thanks to great help here at stackoverflow.

But, I desire to take it one step further.

Instead of one big duplicate free “allofthem.txt” file, how can I modify the above code to create files with a maximum of 5oo lines each from the new data?

They need to go into a new directory eg $dirname.”/done/”.$i.”.txt” I have tried counting in the loop but my efforts are not working and ended up being a mile long.

I also attempted to push 500 into an array, increment to another array and save that way. No luck. I am just not “getting” it.

Again, this beginner needs some expert assistance. Thanks in advance.

Answer 1

Once you have your array of lines as per your code, you can break it into chunks of 500 lines using array_chunk, and then write each chunk to its own file:

// ... from your code
$lines = array_unique($lines);

$counter = 1;
foreach (array_chunk($lines, 500) as $chunk)
{
  file_put_contents($dirname . "/done/" . $counter . ".txt", implode("\n", $chunk));
  $counter++;
}