How to Delete Quotes from Data Read from .Csv File

How to delete quotes from data read from .csv file?

Can you try the following:

import os, fnmatch
import csv
listOfFiles = os.listdir('C:/Users/m/Desktop/csv_files')  
pattern = "*.csv"  
for entry in listOfFiles:  
    if fnmatch.fnmatch(entry, pattern):
        with open(entry, newline='') as csvfile:
            spamreader = csv.reader(csvfile)
            for line in spamreader:
                try:
                    print(list(map(float, line)))
                except:
                    print(line)

Remove double quotes from CSV file

you don't have to iterate lines, you can just read the full content, use replace as you did, and write the new content:

with open('/var/lib/neo4j/import/MRCONSO.csv', "r+", encoding="utf-8") as csv_file:
    content = csv_file.read()

with open('/var/lib/neo4j/import/MRCONSO.csv', "w+", encoding="utf-8") as csv_file:
    csv_file.write(content.replace('"', ''))

How to remove double quotes(") in csv file with Python?

Maybe you could just use replace(...)?

for data in normal:
    data = [d.replace('“', '').replace('”', '') for d in data]
    print(data, len(data))

You can also do lstrip(...) and rstrip(...) if you want to keep internal quotes intact:

for data in normal:
    data = [d.lstrip('“').rstrip('”') for d in data]
    print(data, len(data))

Please note that the quotes you are using are "left double quote" (“) and "right double quote" (”) not just simple "double quote" (").

Remove double quotes if delimiter value is not present in data

This might work for you (GNU sed):

sed ':a;/[^"]$/{N;s/\n//;ba};s/"\([^",]*\)"/\1/g' file

The solution is in two parts:

1. Join broken lines to make whole ones.
2. Remove double quotes surrounding fields that do not contain commas.

If the current line does not end with double quotes, append the next line, remove the newline and repeat. Otherwise: remove double quotes surrounding fields that do not contain double quotes or commas.

N.B. Supposes that fields do not contain quoted double quotes. If that is the case, the condition for the first step would need to be amended and double quotes within fields would need to catered for.

how to remove quotes from csv in php

There really is nothing wrong with the quotes. They avoid any confusion that might occur when some CSV's use whitespace as delimiter:

data    "some more"    another thing
//is not the same as:
data    some more    another thing

However, if you want to remove them, apply this regex to each line:

$line = preg_replace('/(^|;)"([^"]+)";/','$1$2;',$line);

And you should be all right.

How does it work:

• (^|;) matches (and captures) either the beginning of a line, or a semi-colon
• " matches a literal " (doesn't capture)
• ([^" ]+): matches and captures at least one char that is not "
• ";: matches (no capture) a literal " and ;
• $1$2;: the $1 is a back-reference to the first matched group ((^|;))
  
  The $2 references ([^";]+), the ; is just a literal

Suppose $line is '19.08.2013;47657;"12459 Abdullahi";60;', the result (after the preg_replace call) would be: '19.08.2013;47657;12459 Abdullahi;60;'. The quotes are gone.

However, if some field were to contain a " char, it'll probably get escaped (\"), so to prevent the regex from failing to spot that, here's one that uses a lookahead assertion:

$line = preg_replace('/(?<=^|;)"(.+)"(?=;)/','$1',$line);

The difference:

• (?<=^|;) a non-capturing positive lookbehind. The next thing in the pattern will only match if it's preceded either by the beginning of the string (^) or a semi-colon
• (.+) is now the second group. It matches everything, including " BUT:
• "(?=;) this matches a " only if it's followed by a ;.

When presented with a line like '19.08.2013;47657;"12459 \"Abdullahi\"";60;', the latter expression will return 19.08.2013;47657;12459 \"Abdullahi\";60; <-- it only removed the quotes that weren't escaped

Need to remove quotes from text file

In Batch it is easy enough to just remove the quotes and commas and replace the commas with a whitespace

We can therefore just copy the CSV files to a tempfile, then we re-write the content to your actual text file after we replace all the quotes and commas.

@echo off
setlocal enabledelayedexpansion

if exist "\jarvisfs1\groups\InforIFC\Payroll\NEW\tempfile.txt" (del /Q "\jarvisfs1\groups\InforIFC\Payroll\NEW\tempfile.txt")
copy "\jarvisfs1\groups\InforIFC\Payroll\NEW*.csv" "\jarvisfs1\groups\InforIFC\Payroll\NEW\tempfile.txt"
for /f "delims=" %%f in(\jarvisfs1\groups\InforIFC\Payroll\NEW\tempfile.txt) do (
set var=%%f
set var=!var:"=!
set var=!var:,= !
echo !var! >> "\jarvisfs1\groups\InforIFC\Payroll\NEW\EPRO.txt"
)
del /Q "\jarvisfs1\groups\InforIFC\Payroll\NEW\tempfile.txt"

Finally notice how the replace function works. the character you want to replace comes before the = and the character you want to replace it with comes after the = So if you want to replace + with - in a string then you will do !var:+=-! or %var:+=-% if you do not setlocal enabledelayedexpansion to understand exactly how delayedexpansion works, you can run setlocal /? from cmd.exe

How to remove double quotes from index of csv file in python

I think the simpliest is set new column names:

df.columns = ['NoDemande1'] + df.columns[1:].tolist()
print (df)
    NoDemande1     NoUsager Sens IdVehiculeUtilise  NoConducteur    NoAdresse  \
0  42210000003  42210000529    +            265Véh   42210000032  42210002932   
1  42210000005  42210001805    +            265Véh   42210000032  42210002932   
2  42210000004  42210002678    +            265Véh   42210000032  42210002932   
3  42210000003  42210000529    -           265Véh   42210000032  42210004900   
4  42210000004  42210002678    -           265Véh   42210000032  42210007072   
5  42210000005  42210001805    -           265Véh   42210000032  42210004236   

   Fait          HeurePrevue  
0     1  25/07/2015;10:00:04  
1     1  25/07/2015;10:00:04  
2     1  25/07/2015;10:00:04  
3     1  25/07/2015;10:50:03  
4     1  25/07/2015;11:25:03  
5     1  25/07/2015;11:40:03  

Another solution is strip values " from column names:

print (df)
   "NoDemande"     NoUsager Sens IdVehiculeUtilise  NoConducteur    NoAdresse  \
0  42210000003  42210000529    +            265Véh   42210000032  42210002932   
1  42210000005  42210001805    +            265Véh   42210000032  42210002932   
2  42210000004  42210002678    +            265Véh   42210000032  42210002932   
3  42210000003  42210000529    -           265Véh   42210000032  42210004900   
4  42210000004  42210002678    -           265Véh   42210000032  42210007072   
5  42210000005  42210001805    -           265Véh   42210000032  42210004236   

   Fait          HeurePrevue  
0     1  25/07/2015;10:00:04  
1     1  25/07/2015;10:00:04  
2     1  25/07/2015;10:00:04  
3     1  25/07/2015;10:50:03  
4     1  25/07/2015;11:25:03  
5     1  25/07/2015;11:40:03

df.columns = df.columns.str.strip('"')
print (df)
     NoDemande     NoUsager Sens IdVehiculeUtilise  NoConducteur    NoAdresse  \
0  42210000003  42210000529    +            265Véh   42210000032  42210002932   
1  42210000005  42210001805    +            265Véh   42210000032  42210002932   
2  42210000004  42210002678    +            265Véh   42210000032  42210002932   
3  42210000003  42210000529    -            265Véh   42210000032  42210004900   
4  42210000004  42210002678    -            265Véh   42210000032  42210007072   
5  42210000005  42210001805    -            265Véh   42210000032  42210004236   

   Fait          HeurePrevue  
0     1  25/07/2015;10:00:04  
1     1  25/07/2015;10:00:04  
2     1  25/07/2015;10:00:04  
3     1  25/07/2015;10:50:03  
4     1  25/07/2015;11:25:03  
5     1  25/07/2015;11:40:03  

---

Related Topics