Randomly Shuffling Lines in Linux/Bash

How can I shuffle the lines of a text file on the Unix command line or in a shell script?

You can use shuf. On some systems at least (doesn't appear to be in POSIX).

As jleedev pointed out: sort -R might also be an option. On some systems at least; well, you get the picture. It has been pointed out that sort -R doesn't really shuffle but instead sort items according to their hash value.

^{[Editor's note: sort -R almost shuffles, except that duplicate lines / sort keys always end up next to each other. In other words: only with unique input lines / keys is it a true shuffle. While it's true that the output order is determined by hash values, the randomness comes from choosing a random hash function - see manual.]}

Shuffle random list of COMMANDS in BASH?

You can pipe the shuffled commands to bash without any processing:

shuf commands.txt | bash

How can I randomize the lines in a file using standard tools on Red Hat Linux?

And a Perl one-liner you get!

perl -MList::Util -e 'print List::Util::shuffle <>'

It uses a module, but the module is part of the Perl code distribution. If that's not good enough, you may consider rolling your own.

I tried using this with the -i flag ("edit-in-place") to have it edit the file. The documentation suggests it should work, but it doesn't. It still displays the shuffled file to stdout, but this time it deletes the original. I suggest you don't use it.

Consider a shell script:

#!/bin/sh

if [[ $# -eq 0 ]]
then
  echo "Usage: $0 [file ...]"
  exit 1
fi

for i in "$@"
do
  perl -MList::Util -e 'print List::Util::shuffle <>' $i > $i.new
  if [[ `wc -c $i` -eq `wc -c $i.new` ]]
  then
    mv $i.new $i
  else
    echo "Error for file $i!"
  fi
done

Untested, but hopefully works.

bash. How to pick random line from a variable containing numerous lines?

By default, shuf takes a single filename argument, and shuffles the contents of that file. You want it to shuffle its arguments instead; for that, use shuf -e:

b=$(shuf -e -n1 $a)

BTW, there's a subtler problem with this: it'll get confused by filenames with spaces and/or wildcard characters. Maybe not going to happen in your environment, but I prefer to use scripting idioms that don't fall over due to funny filenames. To protect against that, store the filenames in an array rather than counting on word splitting to tell where one stops and the next begins:

readarray -d '' -t arr < <(find ./ -name "*-*.txt" -print0)
b=$(shuf -en1 "${arr[@]}")

If you don't need to store the file list, things are even simpler:

b=$(find ./ -name "*-*.txt" -print0 | shuf -zn1 | tr -d '\0')

---

Related Topics