How to Add Pandas Data to an Existing CSV File

How to add pandas data to an existing csv file?

You can specify a python write mode in the pandas to_csv function. For append it is 'a'.

In your case:

df.to_csv('my_csv.csv', mode='a', header=False)

The default mode is 'w'.

If the file initially might be missing, you can make sure the header is printed at the first write using this variation:

output_path='my_csv.csv'
df.to_csv(output_path, mode='a', header=not os.path.exists(output_path))

How to add pandas data to an existing csv file in Google Cloud Storage?

Google Cloud Storage objects are immutable. This means you cannot modify an object once created. You must implement read-modify-write and replace the existing object.

Object immutability

Objects are immutable, which means that an uploaded object cannot
change throughout its storage lifetime. An object's storage lifetime
is the time between successful object creation, such as uploading, and
successful object deletion. In practice, this means that you cannot
make incremental changes to objects, such as append operations or
truncate operations. However, it is possible to replace objects that
are stored in Cloud Storage, and doing so happens atomically: until
the new upload completes, the old version of the object is served to
readers, and after the upload completes the new version of the object
is served to readers. So a single replacement operation simply marks
the end of one immutable object's lifetime and the beginning of a new
immutable object's lifetime.

Google also supports the Compose API. This supports combining two or more objects to result in a new Cloud Storage object.

Composing Objects

With the Compose API, you could upload the append data to a temporary object, then combine the original object with the append object. This would emulate appending to a file.

Appending data to existing CSV file by columns/rows

The smallest fix, as Shubham P. pointed out, is to make sure you are writing a "line" by including a line break, like '\n':

file = open('dogedata.csv', 'a')
file.write(f'{current_time},{num_of_wallets}\n')
file.close()

The far better fix, as martineau pointed out, is to use the csv module: it's standard, well-developed and used, and importantly takes care of issues like escaping and quoting characters, and it only takes one more line:

file = open('dogedata.csv', 'a', newline='')
writer = csv.writer(f)
writer.writerow([current_time, num_of_wallets])
file.close()

Note that newline='' was added to the open() function, which tells open() not to handle newlines, and instead defer to the writer which has CSV-specific smarts for dealing w/newlines.

Also, instead of using string formatting, you just wrap your data in a list ([current_time, num_of_wallets]); the writer will convert everything to strings/text.

pandas - add additional column to an existing csv file

You can't append columns to a csv file without loading it entirely (however, you can append rows). Use pd.concat:

pd.concat([pd.read_csv('test.csv'), df], axis=1) \
  .to_csv('test.csv', header=True, index=False)

# test.csv before
Hand,Pose
No,Seating Back
No,Seating Back


# test.csv after
Hand,Pose,Eye
No,Seating Back,Left
No,Seating Back,Right

Adding data to csv using pandas dataframe, by adding new column

Add the following and try it out:

records=cur.fetchall()

# Create a dataframe of the SQL query's result 
column_names = ['emp_name','login_count']
df = pd.DataFrame(records, columns = column_names)
df.head()

Now create another dataframe for the daily login counts csv file

df_daily = pd.read_csv('<INSERT the path_to_csv here>')
df_daily.head()

Merge the two dataframes on the 'emp_name' column

result = df.merge(df_daily, on='emp_name')
result.head()

After the join, you can rename the 'login_count' column to today's date

result.rename(columns = {'login_count':'< INSERT date here>'}, inplace = True)

You can then save the new data into a csv file again:

pd.to_csv('<INSERT name of file.csv>', index=False)

Is there a way to append rows to an existing csv file while retaining the colors of rows?

The previous answers cover merging csv. Your question however was about coloring information which probably got ignored up to now, since it makes no sense.
If you are hung up on coloring - you need a different format than csv. csv does not include any formatting information: font, color, width of columns, height of rows, etc none of that is part of a normal csv.

---

Related Topics