Reading very large files in PHP
Are you sure that it's fopen
that's failing and not your script's timeout setting? The default is usually around 30 seconds or so, and if your file is taking longer than that to read in, it may be tripping that up.
Another thing to consider may be the memory limit on your script - reading the file into an array may trip over this, so check your error log for memory warnings.
If neither of the above are your problem, you might look into using fgets
to read the file in line-by-line, processing as you go.
$handle = fopen("/tmp/uploadfile.txt", "r") or die("Couldn't get handle");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
// Process buffer here..
}
fclose($handle);
}
Edit
PHP doesn't seem to throw an error, it just returns false.
Is the path to $rawfile
correct relative to where the script is running? Perhaps try setting an absolute path here for the filename.
Reading huge file line by line in PHP
use fseek, fgets
$handle = fopen("/tmp/uploadfile.txt", "r") or die("Couldn't get handle");
if ($handle) {
while (!feof($handle)) {
$line = fgets($handle);
// Process line here..
}
fclose($handle);
}
Reading very large files in PHP
Read and parse contents of very large file
Yes, you can read it line by line:
$handle = @fopen("/tmp/inputfile.txt", "r");
if ($handle) {
while (($buffer = fgets($handle, 4096)) !== false) {
echo $buffer;
}
fclose($handle);
}
Reading large files from end
It's not pure PHP, but the common solution is to use the tac command which is the revert of cat
and loads the file in reverse. Use exec() or passthru() to run it on the server and then read the results. Example usage:
<?php
$myfile = 'myfile.txt';
$command = "tac $myfile > /tmp/myfilereversed.txt";
exec($command);
$currentRow = 0;
$numRows = 20; // stops after this number of rows
$handle = fopen("/tmp/myfilereversed.txt", "r");
while (!feof($handle) && $currentRow <= $numRows) {
$currentRow++;
$buffer = fgets($handle, 4096);
echo $buffer."<br>";
}
fclose($handle);
?>
Related Topics
Understanding the "Post/Redirect/Get" Pattern
PHP: Running Scheduled Jobs (Cron Jobs)
List of Big-O For PHP Functions
How to Convert a String to a Number in PHP
How to Measure the Speed of Code Written in PHP
Streaming a Large File Using PHP
PHP Exec Command (Or Similar) to Not Wait For Result
Send File Attachment from Form Using PHPmailer and PHP
Is Closing the MySQL Connection Important
How to Set Order by Params Using Prepared Pdo Statement
Pdo Valid Characters For Placeholders
Why Is PHP Printing My Number in Scientific Notation, When I Specified It as .000021
Build Select Query With Dynamic Number of Like Conditions as a MySQLi Prepared Statement
How to Write SQL For a Table That Shares the Same Name as a Protected Keyword in MySQL
How to Get the File Extension in PHP
Laravel: Error [Pdoexception]: Could Not Find Driver in Postgresql