Float64 with Pandas To_Csv

float64 with pandas to_csv

As mentioned in the comments, it is a general floating point problem.

However you can use the float_format key word of to_csv to hide it:

df.to_csv('pandasfile.csv', float_format='%.3f')

or, if you don't want 0.0001 to be rounded to zero:

df.to_csv('pandasfile.csv', float_format='%g')

will give you:

Bob,0.085
Alice,0.005

in your output file.

For an explanation of %g, see Format Specification Mini-Language.

How to use pandas to_csv float_format?

Your code looks fine. Most likely, there is an issue with your input data. Use pd.DataFrame.dtypes to check all your input series are of type float. If they aren't convert to float via:

df[col_list] = df[col_list].apply(pd.to_numeric, downcast='float').fillna(0)

Here's a working example:

from io import StringIO
import pandas as pd

mystr = StringIO("""0.02506602 0.05754493 0.36854688
0.02461631 0.0599653 0.43078098
0.02502534 0.06209149 0.44955311
0.4267356675182389 0.1718682822340447 0.5391386354945895
0.426701667727433 0.17191008887193007 0.5391897818631616
0.4266676661681287 0.17195189807522643 0.5392409104354972""")

df = pd.read_csv(mystr, delim_whitespace=True, header=None)

print(df.dtypes)

# 0    float64
# 1    float64
# 2    float64
# dtype: object

file_loc = r'C:\temp\test.dat'
df.to_csv(file_loc, sep=' ', index=False, header=False, float_format="%.8f")

df = pd.read_csv(file_loc, delim_whitespace=True, header=None)

print(df[0].iloc[-1])

# 0.42666767

Write pandas dataframe to_csv in columns with trailing zeros

You can get the string representation of the dataframe using df.to_string() (docs). Then simply write this string to a text file.

This method also has col_space parameter to further adjust the spacing between columns.

Example:

with open('out.txt', 'w') as file:
    file.writelines(df_rounded.to_string(header=False))

Outputs:

0  1.0  3.000  5.00000
1  1.5  3.455  5.45454

---

There is pandas df.to_csv(float_format='%.5f') (more info) but it applies the formatting to values in all columns. Whereas, in your case, you need different formatting for each column.

Prevent trailing zero with pandas to_csv

Another way using Pandas replace :

df = df.astype(str)
df = df.replace(to_replace = "\.0+$",value = "", regex = True)

This way you don't need to import any extra module.

Output different precision by column with pandas.DataFrame.to_csv()?

Change the type of column "vals" prior to exporting the data frame to a CSV file

df_data['vals'] = df_data['vals'].map(lambda x: '%2.1f' % x)

df_data.to_csv(outfile, index=False, header=False, float_format='%11.6f')

Pandas save to_csv format not tabbing spaces

This is not the sep='\t fault, it is just how the file inside the data is displayed or "how it looks" to our eyes. The file itself is already separated by tabs. This is likely due to the number of digits behind decimal point being different.

I would recommend the following code to make all of it standardized (e.g. 3 digits behind the decimal point)

data.reindex(columns = ['n','Burn_Gain','V','I','I_smooth']).to_csv('pandasfile.csv', float_format='%.3f')

Python Pandas read_csv dtype fails to covert string to float64

data = pd.read_csv(r'\test1.csv', dtype = {'col1': 'float64'}, na_values=[r'/N'])

According to the docs, the na_values parameter is a list-like structure of strings that can be recognised as NaN.

---

Related Topics