Manually merge two files using diff
"I want to output the entire file in a unified format. Is there any way diff can do this?"
Yes.
diff -U 9999999 file1.txt file2.txt > diff.txt
This should work, provided your files are less than 10 million lines long.
Merge 2 text files with differences marked as conflicts
Following the suggestions from the comments, I came up with this one liner which seems to solve my issue.
I would have liked to use constants for the markers in the sed
substitutions, but apparently it's not straightforward to use variables containing \n
with sed
for mac os.
This code seems to work correctly even in docker alpine:3.8 using diffutils.
Other options (brew gnu-sed and similar) might not be easily portable.
diff -D AAAAAAA "${path}" <(echo -n "$decrypted") | \
sed -e $'s/#ifndef AAAAAAA/<<<<<<< file-on-disk/g' | \
sed -e $'s/#endif \/\* ! AAAAAAA \*\//=======\\\n>>>>>>> file-from-secret/g' | \
sed -e $'s/#else \/\* AAAAAAA \*\//=======/g' | \
sed -e $'s/#ifdef AAAAAAA/<<<<<<< file-on-disk\\\n=======/g' | \
sed -e $'s/#endif \/\* AAAAAAA \*\//>>>>>>> file-from-secret/g';
Explanation:
diff -D AAAAAAA "${path}" <(echo -n "$decrypted")
: outputs a merged text with '#ifdef NAME' diffs. I'm using AAAAAAA as marker name. Diff uses #ifndef AAAAAAA
and #endif /* ! AAAAAAA */
to surround text only present in the first file and /#ifdef AAAAAAA
and #endif /* AAAAAAA */
to surround text only present in the second. Note the n
in the first #ifndef
and the !
in the first #endif
comment. As all markers are different, it becomes easy to perform substitutions.
sed -e $'s/#endif \/\* ! AAAAAAA \*\//=======\\\n>>>>>>> file-from-secret/g'
: substitutes the marker with
=======
>>>>>>> file-from-secret
As there is a \n
, the substitution string is enclosed with $''
which interprets correctly the new line character. However, the \
needs to be double-escaped.
How does git compare two files while merging?
There are several merge strategies. 3-way merge algorithm recurse is used by default in Git.
3-way algorithm uses last common commit.
For example:
master: A -> B -> C
Create new branch
master: A -> B -> C
\
branch: D
Some new commits
master: A -> B -> C -> E
\
branch: D -> F
Assume all changes made in a.txt (empty cell corresponds to empty line)
commit C commit E commit F
---------- ---------- ----------
line a line a
line b new line d
line c new line e
line a line b
line b new line f
line c
new line g line c
What happens if we merge two branch (commit E, commit F). Does it produce a merge conflict?. Answer is no. Because git does not compare a file line by line. It compares context of the lines.
Align the a.txt file
commit C commit E commit F
---------- ---------- ----------
new line d
line a-----------line a-----------line a
new line e
line b-----------line b-----------line b
new line f
line c-----------line c-----------line c
new line g
In the above table, changes are aligned. lines in the commit C (ancestor commit) are our references. git compares the neighbor of the reference lines. In the example, we have 4 slot:
- above the line a : commit e adds new line d
- below the line a : commit f adds new line e
- below the line b : commit e adds new line f
- below the line c : commit g adds new line g
As you see, only one of the branches (commit E, commit F) may add something new or both of them may add same thing. Otherwise, A merge conflict is occurred.
How to merge specific file in git and manually resolve diffs?
The git merge
command deals in commits (period, full stop, end of options ... well, there are some options, but they still apply to the commit-wide merge). It:
- starts with the current (
HEAD
) commit; - takes something to specify the other commit, usually a branch name;
- automatically finds the merge-base commit using
git merge-base --all HEAD other-commit
; - and (assuming a single merge base) merges all the files resulting from two diffs:
diff -M base HEAD
, anddiff -M base other
.
So unless one or both of those two commit-wide git diff
commands only pick out one changed file, you cannot use git merge
to do this.
You can use git checkout -p
to do an interactive patch to one file, as compared to a second file. This is not a merge, although some people refer to this as a "two-way merge" operation.
So, the probable solution
You can also use git checkout
or git show
to extract specific versions of a single file from particular commits. If you select three such versions—a merge base, the current commit, and some other commit, for instance—you can then instruct Git to perform a three-way merge on that one file, using the git merge-file
command. For instance:
git show 1234567:foo.txt > /tmp/foo.txt.base
cp foo.txt /tmp/foo.txt.merge
git show 8888888:foo.txt > /tmp/foo.txt.other
(cd /tmp; git merge-file foo.txt.merge foo.txt.base foo.txt.other)
You now have a merged file in /tmp/foo.txt.merge, complete with any conflict markers. Note that this file is independent of a Git repository, but if you like it, you can replace any of the files in your work-tree (such as foo.txt
) with it. Or, of course, you can even just do the merge-file operation directly in your work-tree.
If you replace the first hard-coded commit hash with the other branch name, and the middle (merge base) commit hash with the hash produced by git merge-base --all HEAD otherbranch
, you probably get the kind of thing you want.
A possibly-easier alternative
You can also run git merge -n
to run a full merge (of all files, automatically finding the merge base commit) but avoid committing the resulting merge even if it all seems to Git to have worked. You can then finish only the part of the merge you care about, save the result, run git merge --abort
to terminate the in-progress merge, and copy the saved result back.
How to merge two files using Diff on some selected points?
You can use KDiff3 to do the merge operation. By default it will accept the changes done by both of your, but you can interactively select which version you want. If the common base you both have been working with is named original.c
, your version your.c
and your friend's friend.c
, then the command
kdiff3 -o out.c original.c your.c friend.c
will start an interactive merge where you can select which of the changes you want to have or not.
How to merge files(differentiate) in visual studio code
First using vscode natively with the git toolset
(Make sure to look on the second title as it's a better native way!)
This way may be available on older version of vscode too! Still a good thing to know! (even we should always run on the latest version! And vscode is always keeping getting better and better).
A native powerful and cool way is to use the git toolset within vscode! It still not the most fluid way! But if you are in a setup where you don't have anything else or time or resources to use anything else! Also as a requirement you need to have a git repo initiated! Here we go:
First we will use the change and diffing capability of the git tool set. And the steps go as bellow:
Commit all the current changes
once done: copy past the other file to diff on the place of the current one. And save.
Cool now in the git pallet you can see the file in changes list! Click on it and the diffing editor will show!
Bingo this is it! You can compare and make direct changment! The diffing will keep happening in real time. Note the current state is in the right. And you make changement there.
Here an illustration of direct modification for instance the part in the left is missing from the current file
And here another illustration (current have in plus)
Well to sum up! Git tool and diffing in vscode is so powerful! And all that one need! The only problem is the extra step of committing and cleaning after if desired!
Here some tips! If you want to have the commit history cleaner! Or not have a merge separate! You can remove the last commits from history as much as you need: Without hard reset and commit again a cleaner one!
git reset --soft HEAD~1
You can check
How to cancel a local git commit
Otherwise it can be ok with atomic commit and merge mention!
Also if what you need is to be able to keep a lot from the current file! You can copy the current elsewhere! past the other file to compare! commit ! and then past again the old one! You'll have the old in the right and as current (Not as described on the above) In such a scenario this work well! (Hacky a bit but you may need it).
Native way (direct open of the compare editor)
(May require the newer version of vscode)
open a file that you gonna compare
open the command pallet
CTRL + SHIFT + P
type file: compare
You can see the different possible ways! For a file we can choose compare active file with
.
Then you choose the file! The file need to be within the project directory.
And then you choose a file and the compare editor will open
The above was tested on my brother computer on a new vscode installation. I wasn't sure at first if it was part of the core! And i just confirmed that it is. That too remove the need to the method above involving git! And it's the best native way to go with.
Vscode extensions
Here two extensions i suggest the first:
https://marketplace.visualstudio.com/items?itemName=jinsihou.diff-tool
Easy and simple! It add two elements to the right click menu:
In current file right click -> Select as first file for diff, select one again to view the diff results
select to compare and compare with select no more simple then that !
Another extension to check:
https://marketplace.visualstudio.com/items?itemName=fabiospampinato.vscode-diff
I prefer the first! As this one compare a lot to the native way. And having the control in the contextual menu is just great.
Out of vscode! Using other tools
A quick google search and you'll find a lot of tools!
https://meldmerge.org/
meld merge is cross platform and open source and nice!
in linux and debian:
sudo apt install meld
Otherwise you can check the long list here:
https://www.jotform.com/blog/25-useful-document-and-file-comparison-tools/
https://stackify.com/code-merge-tools/
There is too winMerge to mention (an open source project for windows)
https://winmerge.org/
run git merge algorithm on two individual files
This doesn't really make sense, because you're not providing a common ancestor. If you do have one, however, you can use:
git merge-file <current-version> <common-ancestor> <other-version>
This places the results in the current version file; if you want them elsewhere, use:
git merge-file -p <current> <common> <other> > <dest>
It needs the common ancestor to provide something to consider changes relative to. You could hack it, by providing an empty file or a copy of an older version of one of them from the history of your repository, but the quality of results will depend on how well you select a common ancestor, since it's merging the two diffs, between that and each of the new versions. An empty file will only work well if the two are very similar (many runs of at least three identical lines).
Without that, all you can really do is look at the differences:
git diff --no-index file1 file2
Related Topics
How to File Split at a Line Number
Where Are All My Inodes Being Used
Why Docker Has Ability to Run Different Linux Distribution
What's an Alternative for Dtrace on Linux
Portable Way to Get File Size (In Bytes) in the Shell
How to Set Limit on Directory Size in Linux
Userspace VS Kernel Space Driver
Remove Empty Lines in a Text File via Grep
Pass Output as an Argument for Cp in Bash
Changing Location of Core Dump
Linux Command Line Howto Accept Pairing for Bluetooth Device Without Pin
Walking Page Tables of a Process in Linux
How to Capture All of My Compiler's Output to a File
How to Start Tomcat with Output on Console in Linux
What Is a Way to Read Man Pages in Vim Without Using Temporary Files
Parameter for Shell Scripts That Is Started with Qsub