Tuesday 14 June 2016

algo write a bulk of data into a file faster and more efficient

//we only call fwrite() when the num of lines reach CHUNK.
/call fwrite() every time may largely decrease the writing performance

$totalCount = TOTAL;
$CHUNK = 10000; //No, lines write each time
$counter = 1;


while (false !== $row = mysql_fetch_row($rs)){
$data .= implode("\t", $row)."\n";
if ($counter%$CHUNK == 0||$counter==$totalCount){
echo "proc ".$counter.' ...'."\n";
fwrite($fh_save, $data);
$data = '';
}
$counter++;
}