Utf-8 Problems While Reading CSV File with Fgetcsv

UTF-8 problems while reading CSV file with fgetcsv

Now I got it working (after removing the header command). I think the problem was that the encoding of the php file was in ISO-8859-1. I set it to UTF-8 without BOM. I thought I already have done that, but perhaps I made an additional undo.

Furthermore, I used SET NAMES 'utf8' for the database. Now it is also correct in the database.

fgetcsv() unable to recognize comma after UTF-8 character?

According to a note in the manual, "The locale settings are taken into account by this function. If LC_CTYPE is e.g. en_US.UTF-8, files in one-byte encodings may be read wrongly by this function."

You have the opposite problem. You are trying to read a UTF-8 file but your locale settings use a different encoding. Your LC_CTYPE of "Chinese (Simplified)_China.936" uses the Code page 936 (Simplified Chinese) encoding so fgetcsv is unable to open UTF-8 files unless you change your locale.

You can do this on a per-process basis with (for example) setlocale(LC_ALL, 'en_US.UTF-8'); if the en_US.UTF-8 locale is installed on your system — but read the warning on the manual page about how this can affect other scripts running on the server — or change the intl.default-locale setting in your php.ini.

Some characters in CSV file are not read during PHP fgetcsv()

fgetcsv() only works on standard ASCII characters; so it's probably "correct" in skipping your square root symbols. However, rather than replacing the checkmarks manually, you could read the file into a string, do a str_replace() on those characters, and then parse it using fgetcsv(). You can turn a string into a file pointer (for fgetcsv) thusly:

$fp = fopen('php://memory', 'rw');
fwrite($fp, (string)$string);
rewind($fp);
while (($line = fgetcsv($fp)) !== FALSE)
...

php fgetcsv - charset encoding problems

Try this:

function convert( $str ) {
    return iconv( "Windows-1252", "UTF-8", $str );
}

public function getRow()
{
    if (($row = fgetcsv($this->_handle, 10000, $this->_delimiter)) !== false) {
        $row = array_map( "convert", $row );
        $this->_line++;
        return $this->_headers ? array_combine($this->_headers, $row) : $row;
    } else {
        return false;
    }
}

PHP fgetcsv() not reading first UTF-8 character - alternatif using fgets

I solved this by opening the file using fopen and fgets instead of fgetcsv() and writing a copy using utf8_encode for each line. Then i use the copy and put that through fgetcsv()

here is my updated code.

function runCSVtoArray() {
    // --> FOR IMPORT
    //Function that converts a CSV file to a PHP array.
    //echo '<span class="success">Chargement du fichier CSV pour importation MYSQL....</span><br />';
    $readCharsPerLine = (JRequest::getVar('charsPerLine') > 0) ? JRequest::getVar('charsPerLine') : 1500; /* Import as of 2012-04-16 seem to have max 800chars per line. 1500 is alot of extra. */
    putenv("LANG=fr_CA.UTF-8");
    setlocale(LC_ALL, 'fr_CA.UTF-8');
    //ini_set("auto_detect_line_endings", true);
    //iconv_set_encoding("internal_encoding", "UTF-8");
    $openfile = $this->imp['importPath'].$this->imp['csvFileName'];
    $utf8File = str_replace('.csv', '_utf8.csv', $openfile);

    if ( file_exists($openfile) ) {
        //echo '<span class="success">Fichier CSV trouvé....</span><br />';

        //rewrite the file in UTF8
        if (JRequest::getVar('encodeutf8')) {
            if (($handle = fopen($openfile, "r")) !== FALSE) {
                $newFileHandle = fopen($utf8File, 'w');     //NEW UTF8 FORMAT
                //fwrite($newFileHandle, "\xEF\xBB\xBF");
                while (($the_line = fgets($handle)) !== FALSE) {
                    fwrite($newFileHandle, utf8_encode($the_line));
                }   //End of while()
            }
            $openfile = $utf8File;
        }

        //echo '<span class="success">Ouverture du fichier : '.$openfile.'</span><br />';
        if (($handle = fopen($openfile, "r")) !== FALSE) {
            //echo '<span class="success">Fichier CSV ouvert... Chargement en cours....</span><br />';
            $row_i=0;
            $this->_importData = array();
            while (($data = fgetcsv($handle, $readCharsPerLine, ";")) !== FALSE) {
            /*while (($the_line = fgets($handle)) !== FALSE) {*/
                //$data = explode(';', $the_line);
                //$debugoutput = implode('; ', $data); echo ( (JRequest::getVar('encodeutf8')) && ( mb_detect_encoding($debugoutput, "UTF-8") == "UTF-8") ) ? utf8_encode($debugoutput).'<br />' : $debugoutput.'<br />';   //Debug2
                //$debugoutput = implode('; ', $data); echo $debugoutput.'<br />';  //Debug2
                $num            = count($data);
                if ($row_i==0) {
                    // TITLE ROW
                    $keyRow = array();
                    $maxItems = count($data);   //Count the number of ";"
                    for ($c=0; $c < $num; $c++) {
                        //Making title array with CSV first line
                        //Key for colum
                        if ( (JRequest::getVar('encodeutf8')) && ( mb_detect_encoding($data[$c], "UTF-8") == "UTF-8") ) {
                            //$data[$c] = utf8_encode($data[$c]);
                            $data[$c] = $data[$c];
                        }
                        if ($data[$c]!="") {
                            $keyRow[$c]=trim($data[$c]);
                            $keyRow[$c]=str_replace('GDWACCENT', '', $keyRow[$c]);  //STRIP GDWACCENT, GDW uTF8 fgetcsv fix
                        }
                        else { $keyRow[$c]=''; }
                    }
                } else {
                    //VALUE ROW...
                    for ($c=0; $c < $num; $c++) {
                        $key = $keyRow[$c];
                        if ( (JRequest::getVar('encodeutf8')) && ( mb_detect_encoding($data[$c], "UTF-8") == "UTF-8") ) {
                            //$data[$c] = utf8_encode($data[$c]);
                            $data[$c] = $data[$c];
                            $data[$c]=str_replace('GDWACCENT', '', $data[$c]);  //STRIP GDWACCENT, GDW uTF8 fgetcsv fix
                        }
                        if ($data[$c]!="") {
                            $this->_importData[$row_i][$key]=trim($data[$c]);
                            $this->_importData[$row_i][$key]=str_replace('GDWACCENT', '', $this->_importData[$row_i][$key]);    //STRIP GDWACCENT, GDW uTF8 fgetcsv fix
                        }
                    }   //End of for()
                }
                $row_i++;
            } //End while()
            //echo 'HERE<br />';
            //gdwprint($this->_importData);
            //exit();
            //echo '<span class="success">Chargement terminer.... Sauvegarde en cours...</span><br />';
            return true;
        } else {
            //Incapable d'ouvrir le fichier d'importation.
            return false;
        }
    } else {
        //FILE NOT FOUND...
        return false;
    }
} // runCSVtoArray()

Processing csv file as UTF-8

When you would display it in a browser you should use valid html and set the meta charset to utf8 too:

<?php
include 'head.php';
?>
<!DOCTYPE html>
<html lang="dk">
<head>
    <meta charset="utf-8"/>
</head>
<body>
<?php

$csv = array_map("str_getcsv", file("translations/dk.csv"));
foreach ($csv as $line){
     $translate["dk"][ $line[0] ] = $line[1];
}if ($line[1] != NULL){
    $line[0] = $line[1];
}
echo $line[0];
fclose($csv);
?>
</body>
</html>

Or using text/plain instead of text/html can help:

header('Content-Type: text/plain; charset=UTF-8');

Hope that helps.

---

Related Topics