How to Merge Two Files

How to merge two files line by line in Bash

You can use paste:

paste file1.txt file2.txt > fileresults.txt

How to merge or combine 2 files into single file

Based on your question, you want to create a new file with the content of both files.

You can use io.Copy to achieve that.

Here is a simple command-line tool implementing it.

package main

import (
"io"
"log"
"os"
)

func main() {
if len(os.Args) != 4 {
log.Fatalln("Usage: %s <zip> <signed> <output>\n", os.Args[0])
}
zipName, signedName, output := os.Args[1], os.Args[2], os.Args[3]

zipIn, err := os.Open(zipName)
if err != nil {
log.Fatalln("failed to open zip for reading:", err)
}
defer zipIn.Close()

signedIn, err := os.Open(signedName)
if err != nil {
log.Fatalln("failed to open signed for reading:", err)
}
defer signedIn.Close()

out, err := os.OpenFile(output, os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
log.Fatalln("failed to open outpout file:", err)
}
defer out.Close()

n, err := io.Copy(out, zipIn)
if err != nil {
log.Fatalln("failed to append zip file to output:", err)
}
log.Printf("wrote %d bytes of %s to %s\n", n, zipName, output)

n, err = io.Copy(out, signedIn)
if err != nil {
log.Fatalln("failed to append signed file to output:", err)
}
log.Printf("wrote %d bytes of %s to %s\n", n, signedName, output)
}

Basically, it open both files you want to merge, create a new one and copy the content of each file to the new file.

Manually merge two files using diff

"I want to output the entire file in a unified format. Is there any way diff can do this?"

Yes.

diff -U 9999999 file1.txt file2.txt > diff.txt

This should work, provided your files are less than 10 million lines long.

Merge two files and add computation and sorting the updated data in python

Here is a sample code to output what you need.
I use the formula below to calculate pct change.
percentage_change = 100*(new-old)/old

If old is 0 it is changed to 1 to avoid division by zero error.

import pandas as pd

def read_file(fn):
"""
Read file fn and convert data into a dict of dict.
data = {pname1: {grp: grp1, pname: pname1, cnt: cnt1, cat: cat1},
pname2: {gpr: grp2, ...} ...}
"""
data = {}
with open(fn, 'r') as f:
for lines in f:
line = lines.rstrip()
grp, pname, cnt, cat = line.split(maxsplit=3)
data.update({pname: {'grp': float(grp.replace(',', '')), 'pname': pname, 'cnt': int(cnt), 'cat': cat}})

return data

def process_data(oldfn, newfn):
"""
Read old and new files, update the old file based on new file.
Save output to text, and csv files.
"""
# Get old and new data in dict.
old = read_file(oldfn)
new = read_file(newfn)

# Update old data based on new data
u_data = {}
for ko, vo in old.items():
if ko in new:
n = new[ko]

# Update cnt.
old_cnt = vo['cnt']
new_cnt = n['cnt']
u_cnt = old_cnt + new_cnt

# cnt change, if old is zero we set it to 1 to avoid division by zero error.
tmp_old_cnt = 1 if old_cnt == 0 else old_cnt
cnt_change = 100 * (new_cnt - tmp_old_cnt) / tmp_old_cnt

# grp change
old_grp = vo['grp']
new_grp = n['grp']
grp_change = 100 * (new_grp - old_grp) / old_grp

u_data.update({ko: {'grp': n['grp'], 'pname': n['pname'], 'cnt': u_cnt, 'cat': n['cat'],
'cnt_change%': round(cnt_change, 2), 'grp_change%': round(grp_change, 2)}})

# add new data to u_data, that is not in old data
for kn, vn in new.items():
if kn not in old:
# Since this is new item its old cnt is zero, we set it to 1 to avoid division by zero error.
old_cnt = 1
new_cnt = vn['cnt']
cnt_change = 100 * (new_cnt - old_cnt) / old_cnt

# grp change is similar to cnt change
old_grp = 1
new_grp = vn['grp']
grp_change = 100 * (new_grp - old_grp) / old_grp

# Update new columns.
vn.update({'cnt_change%': round(cnt_change, 2), 'grp_change%': round(grp_change, 2)})
u_data.update({kn: vn})

# Create new data mydata list from u_data, and only extract the dict value.
mydata = []
for _, v in u_data.items():
mydata.append(v)

# Convert mydata into pandas dataframe to easier manage the data.
df = pd.DataFrame(mydata)
df = df.sort_values(by=['cnt'], ascending=False) # sort on cnt column

# Save to csv file.
df.to_csv('output.csv', index=False)

# Save to text file.
with open('output.txt', 'w') as w:
w.write(f'{df.to_string(index=False)}')

# Print in console.
print(df.to_string(index=False))

# Start
oldfn = 'F:/Tmp/oldFile.txt'
newfn = 'F:/Tmp/newFile.txt'
process_data(oldfn, newfn)

Console output:

   grp                    pname  cnt                      cat  cnt_change%  grp_change%
8739.0 6ea059a29eccecee4e250414 62 MAXIMACASH (MAXCAS...) 2900.00 219.06
138.0 1c6bc8e962427deb4106ae06 58 Charge (Charge) 525.00 -68.49
860.0 31b5c07636dab8f0909dbd2d 46 Buff Unicorn (BUFFUN...) 566.67 272.29
200.0 9e4d81c8fc15870b15aef8dc 33 BABY BNB (BBNB) 900.00 -28.32
20.0 5esdsds2sd15870b15aef8dc 30 CharliesAngel (CA) 2900.00 1900.00
1560.0 c15e89f2149bcc0cbd5fb204 24 HUH_Token (HUH) 400.00 11.75

text output:

   grp                    pname  cnt                      cat  cnt_change%  grp_change%
8739.0 6ea059a29eccecee4e250414 62 MAXIMACASH (MAXCAS...) 2900.00 219.06
138.0 1c6bc8e962427deb4106ae06 58 Charge (Charge) 525.00 -68.49
860.0 31b5c07636dab8f0909dbd2d 46 Buff Unicorn (BUFFUN...) 566.67 272.29
200.0 9e4d81c8fc15870b15aef8dc 33 BABY BNB (BBNB) 900.00 -28.32
20.0 5esdsds2sd15870b15aef8dc 30 CharliesAngel (CA) 2900.00 1900.00
1560.0 c15e89f2149bcc0cbd5fb204 24 HUH_Token (HUH) 400.00 11.75

csv output:

grp,pname,cnt,cat,cnt_change%,grp_change%
8739.0,6ea059a29eccecee4e250414,62,MAXIMACASH (MAXCAS...),2900.0,219.06
138.0,1c6bc8e962427deb4106ae06,58,Charge (Charge),525.0,-68.49
860.0,31b5c07636dab8f0909dbd2d,46,Buff Unicorn (BUFFUN...),566.67,272.29
200.0,9e4d81c8fc15870b15aef8dc,33,BABY BNB (BBNB),900.0,-28.32
20.0,5esdsds2sd15870b15aef8dc,30,CharliesAngel (CA),2900.0,1900.0
1560.0,c15e89f2149bcc0cbd5fb204,24,HUH_Token (HUH),400.0,11.75

How to merge multiple files horizontally in a single file with multiple columns?

file_list = ['File1.txt', 'File2.txt', 'File3.txt']
df = pd.DataFrame()
for file in file_list:
temp_df = pd.read_csv(file, header=None, names=[file[:-4]])
df = pd.concat([df, temp_df], axis=1)

print(df)

File1 File2 File3
0 -0.000633 -0.003002 -0.002638
1 -0.001362 -0.003184 -0.000086
2 -0.001909 -0.004096 0.001736
3 -0.002638 -0.003913 0.001736
4 -0.002820 NaN 0.001736
5 -0.004096 NaN NaN


Related Topics



Leave a reply



Submit