Linux: Create Random Directory/File Hierarchy

Linux: create random directory/file hierarchy

I wasn't too happy with the given answers, so I came up with my own. The following takes my input files and uses /dev/urandom to gather 10 to 256 printable chars, puts in a few more directory separators, creates the directory hierarchy and places a file in it.

Using urandom creates some really weird directory names which is good for my purpose. I'm sure a real Unix guru could simplify this even more. The dir building could probably be done in a single awk command for example.

#!/bin/bash
INDIR='files';

IFS=$'\n'
for FILE in `ls $INDIR/*`; do
    DIR=`cat /dev/urandom | \
         tr -dc '[ -~]' | \
         tr 'ABCDEF\\\\' '///////' | \
         head -c$((10 + $RANDOM % 256))`

    mkdir -p $DIR
    cp $FILE $DIR
done

How can I select random files from a directory in bash?

Here's a script that uses GNU sort's random option:

ls |sort -R |tail -$N |while read file; do
    # Something involving $file, or you can leave
    # off the while to just get the filenames
done

Generate random directories/files given number of files and depth

Why not download some real open source repos and use those?

Have you thought about what goes into the files? is that random data too?

How to create a temporary directory?

Use mktemp -d. It creates a temporary directory with a random name and makes sure that file doesn't already exist. You need to remember to delete the directory after using it though.

Deleting files in specific sub-directories created at random

Assuming you are in the top level directory where you want to start searching for snapshot folders (note - replaced your '-mtime 5 -exec rm' with just a -ls so you can verify it is finding the folders properly):

find . -name snapshot -a -type d | while read a ; do find "$a" -ls ; done

or, even easier:

find . -path '*/snapshot/*' -a -type f -a -mtime 5 -exec ls -l {} \;

Select randomly a file from various directories and sort

This shuffles all the files in the source tree, partially sorts on the numeric part with a stable sort, so the other elements remain shuffled.

$ target=~/tmp/shuf
$ destination=/filepath/
$ tree $target
~/tmp/shuf
`-- papers
    |-- semester_1
    |   |-- cs630-linux_research_paper-fname_lname-001.txt
    |   |-- cs635-progamming_languages-fname_lname-002.txt
    |   |-- cs645-java_programming_paper-fname_lname-003.txt
    |   `-- cs900-computer_robotics_capstone-fname_lname-004.txt
    |-- semester_2
    |   |-- cs650-software_methodologies-fname_lname-001.txt
    |   |-- cs675-nosql_db_research-fname_lname-002.txt
    |   |-- cs700-artificial_intelligence_reasearch-fname_lname-003.txt
    |   |-- cs800-algorithms_and_computational_complexity-fname_lname-004.txt
    |   |-- cs825-database_systems_internals-fname_lname-005.txt
    |   `-- cs850-computer_graphics-fname_lname-006.txt
    `-- semester_3
        |-- cs725-web_programming_technologies-fname_lname-001.txt
        |-- cs750-data_programming-fname_lname-002.txt
        `-- cs775-hardware_software_interface_paper-fname_lname-003.txt

4 directories, 13 files
$ find $target -type f -iname "*.txt" \
   | shuf \
   | awk -F- '{printf("%s:%s\n", $0, $NF)}' \
   | sort -t : -k 2 -s \
   | cut -d : -f 1 \
   | xargs -n1 basename \
   | sed "s,^,$destination,"
/filepath/cs725-web_programming_technologies-fname_lname-001.txt
/filepath/cs650-software_methodologies-fname_lname-001.txt
/filepath/cs630-linux_research_paper-fname_lname-001.txt
/filepath/cs635-progamming_languages-fname_lname-002.txt
/filepath/cs750-data_programming-fname_lname-002.txt
/filepath/cs675-nosql_db_research-fname_lname-002.txt
/filepath/cs775-hardware_software_interface_paper-fname_lname-003.txt
/filepath/cs700-artificial_intelligence_reasearch-fname_lname-003.txt
/filepath/cs645-java_programming_paper-fname_lname-003.txt
/filepath/cs900-computer_robotics_capstone-fname_lname-004.txt
/filepath/cs800-algorithms_and_computational_complexity-fname_lname-004.txt
/filepath/cs825-database_systems_internals-fname_lname-005.txt
/filepath/cs850-computer_graphics-fname_lname-006.txt

To store the result in a file called filename, you can redirect:

$ find $target -type f -iname "*.txt" \
   | shuf \
   | awk -F- '{printf("%s:%s\n", $0, $NF)}' \
   | sort -t : -k 2 -s \
   | cut -d : -f 1 \
   | xargs -n1 basename \
   | sed "s,^,$destination," \
   > filename

How to create a temporary directory in C in Linux?

I suggest to use the mkdtemp() function together with usual functions from the C API (glibc). Here is a full answer:

EDIT: The answer from Nemanja Boric is unfortunately not usable in practice because the rmdir() function is only intended to remove an empty directory. Here is a full correct answer:

#define  _POSIX_C_SOURCE 200809L
#define  _XOPEN_SOURCE 500L

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#include <errno.h>
#include <ftw.h>

/* Call-back to the 'remove()' function called by nftw() */
static int
remove_callback(const char *pathname,
                __attribute__((unused)) const struct stat *sbuf,
                __attribute__((unused)) int type,
                __attribute__((unused)) struct FTW *ftwb)
{
  return remove (pathname);
}

int
main ()
{
  /* Create the temporary directory */
  char template[] = "/tmp/tmpdir.XXXXXX";
  char *tmp_dirname = mkdtemp (template);

  if (tmp_dirname == NULL)
  {
     perror ("tempdir: error: Could not create tmp directory");
     exit (EXIT_FAILURE);
  }

  /* Change directory */
  if (chdir (tmp_dirname) == -1)
  {
     perror ("tempdir: error: ");
     exit (EXIT_FAILURE);
  }

  /******************************/
  /***** Do your stuff here *****/
  /******************************/

  /* Delete the temporary directory */
  if (nftw (tmp_dirname, remove_callback, FOPEN_MAX,
            FTW_DEPTH | FTW_MOUNT | FTW_PHYS) == -1)
    {
      perror("tempdir: error: ");
      exit(EXIT_FAILURE);
    }

  return EXIT_SUCCESS;
}

---

Related Topics