Question

I downloaded a database that was exported to the TXT format and has about 700MB with 7 million records (1 per line).
I made a script to import the data to a mysql database, but when about 4 million records inserted into, the browser crashes.
I have tested in Firefox and IE.
Can someone give me an opinion and some advice about this?

The script is this:

<?php
set_time_limit(0);
ini_set('memory_limit','128M');

$conexao = mysql_connect("localhost","root","") or die (mysql_error());
$base = mysql_select_db("lista102",$conexao) or die (mysql_error());
$ponteiro = fopen("TELEFONES_1.txt","r");
$conta = 0;
function myflush(){ ob_flush(); flush(); }

while(!feof($ponteiro)){
    $conta++;

    $linha = fgets($ponteiro,4096);
    $linha = str_replace("\"", "", $linha);
    $arr = explode(";",$linha);
    $sql = "insert into usuarios (CPF_CNPJ,NOME,LOG,END,NUM,COMPL,BAIR,MUN,CEP,DDD,TELEF) values ('".$arr[0]."','".$arr[1]."','".$arr[2]."','".$arr[3]."','".$arr[4]."','".$arr[5]."','".$arr[6]."','".$arr[7]."','".$arr[8]."','".$arr[9]."','".trim($arr[10])."')";
    $rs = mysql_query($sql);
    if(!$rs){ echo $conta ." error";}

    if(($conta%5000)==4999) { sleep(10); echo "<br>Pause: ".$conta; }
    myflush();
}

echo "<BR>Eof, import complete";
fclose($ponteiro);
mysql_close($conexao);
?>

Answer 1

Try splitting the file in 100 MB chunks. This is a quick solving suggestion to get the job done. The browser issue can get complicated to solve. Try also different browsers.

phpMyadmin has options to continue the query if a crash happened. Allows interrupt of import in case script detects it is close to time limit. This might be good way to import large files, however it can break transactions.