Compare Two Different Urls Using Linux

Compare two different urls using Linux

Of course you can use curl, although it is a bit strange:

diff <(curl url1) <(curl url2)

In your case:

diff <(curl http://www.example.net/index.php) <(curl http://www.example.com/index.php)

Note you can use curl -s for a cleaner diff execution.

Compare two websites and see if they are equal?

Get the formatted output of both sites (here we use w3m, but lynx can also work):

w3m -dump http://google.com 2>/dev/null > /tmp/1.html
w3m -dump http://google.de 2>/dev/null > /tmp/2.html

Then use wdiff, it can give you a percentage of how similar the two texts are.

wdiff -nis /tmp/1.html /tmp/2.html

It can be also easier to see the differences using colordiff.

wdiff -nis /tmp/1.html /tmp/2.html | colordiff

Excerpt of output:

Web Images Vidéos Maps [-Actualités-] Livres {+Traduction+} Gmail plus »
[-iGoogle |-]
Paramètres | Connexion

                           Google [hp1] [hp2]
                                  [hp3] [-Français-] {+Deutschland+}

           [                                                         ] Recherche
                                                                       avancéeOutils
                      [Recherche Google][J'ai de la chance]            linguistiques

/tmp/1.html: 43 words  39 90% common  3 6% deleted  1 2% changed
/tmp/2.html: 49 words  39 79% common  9 18% inserted  1 2% changed

(he actually put google.com into french... funny)

The common % values are how similar both texts are. Plus you can easily see the differences by word (instead of by line which can be a clutter).

Compare two websites and see if they are equal?

Get the formatted output of both sites (here we use w3m, but lynx can also work):

w3m -dump http://google.com 2>/dev/null > /tmp/1.html
w3m -dump http://google.de 2>/dev/null > /tmp/2.html

Then use wdiff, it can give you a percentage of how similar the two texts are.

wdiff -nis /tmp/1.html /tmp/2.html

It can be also easier to see the differences using colordiff.

wdiff -nis /tmp/1.html /tmp/2.html | colordiff

Excerpt of output:

Web Images Vidéos Maps [-Actualités-] Livres {+Traduction+} Gmail plus »
[-iGoogle |-]
Paramètres | Connexion

                           Google [hp1] [hp2]
                                  [hp3] [-Français-] {+Deutschland+}

           [                                                         ] Recherche
                                                                       avancéeOutils
                      [Recherche Google][J'ai de la chance]            linguistiques

/tmp/1.html: 43 words  39 90% common  3 6% deleted  1 2% changed
/tmp/2.html: 49 words  39 79% common  9 18% inserted  1 2% changed

(he actually put google.com into french... funny)

The common % values are how similar both texts are. Plus you can easily see the differences by word (instead of by line which can be a clutter).

How to compare 2 symbolic links in unix (Linux)?

For GNU systems (and possibly others, but I can't say), there's readlink(1):

$ touch a
$ ln -s a b
$ readlink b
a

You can use that in comparisons:

$ test $(readlink -f a) = $(readlink -f b)
$ echo $?
0

compare contents of two directories on remote server using unix

You can use rsync with the -n flag to find out if the files are in sync, without actually doing a sync.

For example, from server1:

rsync -n -avrc /abc/home/sample1/* server2:/abc/home/sample2/

This will print the names of all files (recursive, with the -r flag) that differ between server1:/abc/home/sample1/ and server2:/abc/home/sample2/

rsync used parameters explanation

-n, --dry-run - perform a trial run with no changes made

-a, --archive - archive mode; equals -rlptgoD (no -H,-A,-X)

-v, --verbose - increase verbosity

-r, --recursive - recurse into directories

-c, --checksum - skip based on checksum, not mod-time & size

Comparing 2 files in linux for different word

Use the "diff" command to compare 2 files:

$ diff a.txt b.txt

Or, for a unified diff:

$ diff -u a.txt b.txt

Use -u0 for a unified diff without context.

Fastest way to tell if two files have the same contents in Unix/Linux?

I believe cmp will stop at the first byte difference:

cmp --silent $old $new || echo "files are different"

Comparing two files in linux terminal

Here is my solution for this :

mkdir temp
mkdir results
cp /usr/share/dict/american-english ~/temp/american-english-dictionary
cp /usr/share/dict/british-english ~/temp/british-english-dictionary
cat ~/temp/american-english-dictionary | wc -l > ~/results/count-american-english-dictionary
cat ~/temp/british-english-dictionary | wc -l > ~/results/count-british-english-dictionary
grep -Fxf ~/temp/american-english-dictionary ~/temp/british-english-dictionary > ~/results/common-english
grep -Fxvf ~/results/common-english ~/temp/american-english-dictionary > ~/results/unique-american-english
grep -Fxvf ~/results/common-english ~/temp/british-english-dictionary > ~/results/unique-british-english

---

Related Topics