How to merge two files line by line in Bash
You can use paste
:
paste file1.txt file2.txt > fileresults.txt
How to merge or combine 2 files into single file
Based on your question, you want to create a new file with the content of both files.
You can use io.Copy to achieve that.
Here is a simple command-line tool implementing it.
package main
import (
"io"
"log"
"os"
)
func main() {
if len(os.Args) != 4 {
log.Fatalln("Usage: %s <zip> <signed> <output>\n", os.Args[0])
}
zipName, signedName, output := os.Args[1], os.Args[2], os.Args[3]
zipIn, err := os.Open(zipName)
if err != nil {
log.Fatalln("failed to open zip for reading:", err)
}
defer zipIn.Close()
signedIn, err := os.Open(signedName)
if err != nil {
log.Fatalln("failed to open signed for reading:", err)
}
defer signedIn.Close()
out, err := os.OpenFile(output, os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
log.Fatalln("failed to open outpout file:", err)
}
defer out.Close()
n, err := io.Copy(out, zipIn)
if err != nil {
log.Fatalln("failed to append zip file to output:", err)
}
log.Printf("wrote %d bytes of %s to %s\n", n, zipName, output)
n, err = io.Copy(out, signedIn)
if err != nil {
log.Fatalln("failed to append signed file to output:", err)
}
log.Printf("wrote %d bytes of %s to %s\n", n, signedName, output)
}
Basically, it open both files you want to merge, create a new one and copy the content of each file to the new file.
Manually merge two files using diff
"I want to output the entire file in a unified format. Is there any way diff can do this?"
Yes.
diff -U 9999999 file1.txt file2.txt > diff.txt
This should work, provided your files are less than 10 million lines long.
Merge two files and add computation and sorting the updated data in python
Here is a sample code to output what you need.
I use the formula below to calculate pct change.percentage_change = 100*(new-old)/old
If old is 0 it is changed to 1 to avoid division by zero error.
import pandas as pd
def read_file(fn):
"""
Read file fn and convert data into a dict of dict.
data = {pname1: {grp: grp1, pname: pname1, cnt: cnt1, cat: cat1},
pname2: {gpr: grp2, ...} ...}
"""
data = {}
with open(fn, 'r') as f:
for lines in f:
line = lines.rstrip()
grp, pname, cnt, cat = line.split(maxsplit=3)
data.update({pname: {'grp': float(grp.replace(',', '')), 'pname': pname, 'cnt': int(cnt), 'cat': cat}})
return data
def process_data(oldfn, newfn):
"""
Read old and new files, update the old file based on new file.
Save output to text, and csv files.
"""
# Get old and new data in dict.
old = read_file(oldfn)
new = read_file(newfn)
# Update old data based on new data
u_data = {}
for ko, vo in old.items():
if ko in new:
n = new[ko]
# Update cnt.
old_cnt = vo['cnt']
new_cnt = n['cnt']
u_cnt = old_cnt + new_cnt
# cnt change, if old is zero we set it to 1 to avoid division by zero error.
tmp_old_cnt = 1 if old_cnt == 0 else old_cnt
cnt_change = 100 * (new_cnt - tmp_old_cnt) / tmp_old_cnt
# grp change
old_grp = vo['grp']
new_grp = n['grp']
grp_change = 100 * (new_grp - old_grp) / old_grp
u_data.update({ko: {'grp': n['grp'], 'pname': n['pname'], 'cnt': u_cnt, 'cat': n['cat'],
'cnt_change%': round(cnt_change, 2), 'grp_change%': round(grp_change, 2)}})
# add new data to u_data, that is not in old data
for kn, vn in new.items():
if kn not in old:
# Since this is new item its old cnt is zero, we set it to 1 to avoid division by zero error.
old_cnt = 1
new_cnt = vn['cnt']
cnt_change = 100 * (new_cnt - old_cnt) / old_cnt
# grp change is similar to cnt change
old_grp = 1
new_grp = vn['grp']
grp_change = 100 * (new_grp - old_grp) / old_grp
# Update new columns.
vn.update({'cnt_change%': round(cnt_change, 2), 'grp_change%': round(grp_change, 2)})
u_data.update({kn: vn})
# Create new data mydata list from u_data, and only extract the dict value.
mydata = []
for _, v in u_data.items():
mydata.append(v)
# Convert mydata into pandas dataframe to easier manage the data.
df = pd.DataFrame(mydata)
df = df.sort_values(by=['cnt'], ascending=False) # sort on cnt column
# Save to csv file.
df.to_csv('output.csv', index=False)
# Save to text file.
with open('output.txt', 'w') as w:
w.write(f'{df.to_string(index=False)}')
# Print in console.
print(df.to_string(index=False))
# Start
oldfn = 'F:/Tmp/oldFile.txt'
newfn = 'F:/Tmp/newFile.txt'
process_data(oldfn, newfn)
Console output:
grp pname cnt cat cnt_change% grp_change%
8739.0 6ea059a29eccecee4e250414 62 MAXIMACASH (MAXCAS...) 2900.00 219.06
138.0 1c6bc8e962427deb4106ae06 58 Charge (Charge) 525.00 -68.49
860.0 31b5c07636dab8f0909dbd2d 46 Buff Unicorn (BUFFUN...) 566.67 272.29
200.0 9e4d81c8fc15870b15aef8dc 33 BABY BNB (BBNB) 900.00 -28.32
20.0 5esdsds2sd15870b15aef8dc 30 CharliesAngel (CA) 2900.00 1900.00
1560.0 c15e89f2149bcc0cbd5fb204 24 HUH_Token (HUH) 400.00 11.75
text output:
grp pname cnt cat cnt_change% grp_change%
8739.0 6ea059a29eccecee4e250414 62 MAXIMACASH (MAXCAS...) 2900.00 219.06
138.0 1c6bc8e962427deb4106ae06 58 Charge (Charge) 525.00 -68.49
860.0 31b5c07636dab8f0909dbd2d 46 Buff Unicorn (BUFFUN...) 566.67 272.29
200.0 9e4d81c8fc15870b15aef8dc 33 BABY BNB (BBNB) 900.00 -28.32
20.0 5esdsds2sd15870b15aef8dc 30 CharliesAngel (CA) 2900.00 1900.00
1560.0 c15e89f2149bcc0cbd5fb204 24 HUH_Token (HUH) 400.00 11.75
csv output:
grp,pname,cnt,cat,cnt_change%,grp_change%
8739.0,6ea059a29eccecee4e250414,62,MAXIMACASH (MAXCAS...),2900.0,219.06
138.0,1c6bc8e962427deb4106ae06,58,Charge (Charge),525.0,-68.49
860.0,31b5c07636dab8f0909dbd2d,46,Buff Unicorn (BUFFUN...),566.67,272.29
200.0,9e4d81c8fc15870b15aef8dc,33,BABY BNB (BBNB),900.0,-28.32
20.0,5esdsds2sd15870b15aef8dc,30,CharliesAngel (CA),2900.0,1900.0
1560.0,c15e89f2149bcc0cbd5fb204,24,HUH_Token (HUH),400.0,11.75
How to merge multiple files horizontally in a single file with multiple columns?
file_list = ['File1.txt', 'File2.txt', 'File3.txt']
df = pd.DataFrame()
for file in file_list:
temp_df = pd.read_csv(file, header=None, names=[file[:-4]])
df = pd.concat([df, temp_df], axis=1)
print(df)
File1 File2 File3
0 -0.000633 -0.003002 -0.002638
1 -0.001362 -0.003184 -0.000086
2 -0.001909 -0.004096 0.001736
3 -0.002638 -0.003913 0.001736
4 -0.002820 NaN 0.001736
5 -0.004096 NaN NaN
Related Topics
Linux Split a Column into Two Different Columns in a Same CSV File
Why Fftw on Windows Is Faster Than on Linux
Docker Overlay2: Error Walking File System: Oserror [Errno 40] Too Many Levels of Symbolic Links
How to Delete Files Over (N) Days Old But Leave (N) Files Regardless of Age
Filter a Content File to Table
Shell Script Error: "Head: Invalid Trailing Option -- 1"
Synchronize Linux System Clock to Windows Ntp Service
Print the Directory Where the 'Find' Linux Command Finds a Match
How to Increase Openfabrics Memory Limit for Torque Jobs
Perf Top Result About Nested Functions
How to Configure Acpi *.Asl for a Virtual Mdio-Gpio Device Connected to a I2C Gpio Expander
Run Bash Script in Raspberry Pi via Ifttt
Stack Smashing Code Not Working on Linux Kernel 2.6.38.7... Please Help
Webpack Build Fails with Gitlab-Ci
How to Print Multiple Strings on New Line in the Assembly Language
Sending Mail Body and Attachment Using Mailx - Linux
Why Does This Shell Pipeline Exit
Use Grep to Find the Words That Have Two 'S' Separated by a Space