Process Large File Using PHP
PHP offers a nice platform and features to the developers. But, still, this programming language comes with numerous limitations. And, one of the biggest limitations of PHP that we will study today, is the limitation of processing large files. If you have tried to open the large file using PHP, then you can understand our point here.
When you try to process large files using PHP regular functions like file_get_contents() or file(), then these limitations can be seen. The prominent limitations that you will face using PHP for large files process are:
Memory Limitation – Above mentioned PHP relies on the memory_limit setting in the php.ini file. You can increase the value but still at one point whole content of the file will be saved on your memory. So, if you are planning to process 20 GB file using PHP, then we advise you to think twice.
Slow User Experience – Okay, so you might have to wait for a good few minutes to get the output result. As the input array takes quite to interpret output result. So, it’s a pretty bad user experience, but can be fixed using yield keywords.
Best Way To Read File Using PHP Line By Line
Well, we might mention hundreds of php limitations, but, still, at one point of your career, you might have to read the file using PHP. Then, what?
You can either exhaust all your memory and get your memory server down or you can be smart. Yep, you see people, PHP also understands the problem of memory server that’s why they have developed numerous inbuilt tools. The tools help PHP user to process the large files, in large files we mean more than 1 GB.
That’s pretty good and you can even read file line by line or in the chunk. It’s all up to you. So, if you are interested in opening the large file with the PHP, then don’t forget to install the following tools first:
#1. Processing a File Line by Line into an Array Using file()
You can easily use file() function in an array to read the full file. This function stores each line along with the newline character in its own array element. So, if you want to read file line by line, then save the line is array and file () is a natural choice.
But, this method won’t work for the too big files. Because it will read the whole content at once. Well, it does solve the problem line by the, but for that, it reads the whole content at once. And, that’s not applicable for the large files. For example, if you have read a text file called The Life of the Bee from Project Gutenberg. It will be no problem as the file is just around 360 KB, but it will be a problem if you have to read 1 GB file.
#2. Processing a Large File One Line at a Time Using fgets ()
In PHP gets () function, the current line is read up to the pointed mark set by the handler. This function won’t read after reaching to the specified limit, whether the end of line or end of the selection. If there is no more data to read, then it returns FALSE. That means we can check whether the file has been completely compared using fgets().
If you haven’t specified the limit, then it will read the full line. In case, you have long lines in the document, then you might fully exhaust your memory. Unless you won’t have long lines in your document, then using fgets() will be the very good solution for your memory.
#3. Processing a Large File in Smaller Pieces Using fread()
The major drawback of using the fgets() function in PHP that it won’t exhaust your memory if the line is too large. Well, this is a very unlikely situation, but we have to be prepared for all the situations.
So, in that case, use fread() function in PHP, if your fgets() function fails. As in the fread() function, file up to length number of bytes can be read. You are the one who gets to specify the length so you don’t have to worry about running out of memory.
So, folks, consider your file size as whole and size of each line before reading the file using PHP. And, according to your file size and line size, pick the PHP function, because there is no smartness in exhausting your memory for nothing.