Did I Understand /Dev/Urandom

Did I understand /dev/urandom?

From the urandom manpage:

The random number generator gathers
environmental noise from device
drivers and other sources into an
entropy pool. The generator also
keeps an estimate of the number of
bits of noise in the entropy pool.
From this entropy pool random numbers
are created.

When read, the /dev/random device
will only return random bytes within
the estimated number of bits of noise
in the entropy pool. /dev/random
should be suitable for uses that need
very high quality randomness such
as one-time pad or key
generation. When the entropy pool
is empty, reads from /dev/random will
block
until additional environmental
noise is gathered.

A read from the /dev/urandom device will not block waiting for more
entropy
. As a result, if there is
not sufficient entropy in the entropy
pool, the returned values are
theoretically vulnerable to a
cryptographic attack on the algorithms
used by the driver. Knowledge of how
to do this is not available in the
current unclassified literature, but
it is theoretically possible that such
an attack may exist. If this is a
concern in your application, use
/dev/random instead.

both uses a PRNG, though using environmental data and entropy pool makes it astronomically much more difficult to crack the PRNG, and impossible without also gathering the exact same environmental data.

As a rule of thumb, without specialized expensive hardware that gathers data from, say, quantum events, there is no such thing as true random number generator (i.e. a RNG that generates truly unpredictable number); though for cryptographic purpose, /dev/random or /dev/urandom will suffice (the method used is for a CPRNG, cryptographic pseudo-random number generator).

The entropy pool and blocking read of /dev/random is used as a safe-guard to ensure the impossibility of predicting the random number; if, for example, an attacker exhausted the entropy pool of a system, it is possible, though highly unlikely with today's technology, that he can predict the output of /dev/urandom which hasn't been reseeded for a long time (though doing that would also require the attacker to exhaust the system's ability to collect more entropies, which is also astronomically improbably).

Reading /dev/urandom as early as possible

urandom is provided via device driver and the first thing kernel does with the driver is to call init call.

If you take a look here: http://lxr.free-electrons.com/source/drivers/char/random.c#L1401

  * Note that setup_arch() may call add_device_randomness()
* long before we get here. This allows seeding of the pools
* with some platform dependent data very early in the boot
* process. But it limits our options here. We must use
* statically allocated structures that already have all
* initializations complete at compile time. We should also
* take care not to overwrite the precious per platform data
* we were given.
*/
static int rand_initialize(void)
{
init_std_data(&input_pool);
init_std_data(&blocking_pool);
init_std_data(&nonblocking_pool);
return 0;
}
early_initcall(rand_initialize);

So, init function for this driver is rand_initialize. However note that comment says that setup_arch may call add_device randomness() before this device is even initialized. However, calling that function does not add any actual entropy (it feeds the pool with stuff like MAC addresses, so if you have two exactly the same VMs, you're good there). From the comment:

  * add_device_randomness() is for adding data to the random pool that
* is likely to differ between two devices (or possibly even per boot).
* This would be things like MAC addresses or serial numbers, or the
* read-out of the RTC. This does *not* add any actual entropy to the
* pool, but it initializes the pool to different values for devices
* that might otherwise be identical and have very little entropy
* available to them (particularly common in the embedded world).

Also, note that entropy pools are stored on shutdown and restored on boot time via init script (on my Ubuntu 14.04, it's in /etc/init.d/urandom), so you might want to call your script from that script before

 53     (
54 date +%s.%N
55
56 # Load and then save $POOLBYTES bytes,
57 # which is the size of the entropy pool
58 if [ -f "$SAVEDFILE" ]
59 then
60 cat "$SAVEDFILE"
61 fi
62 # Redirect output of subshell (not individual commands)
63 # to cope with a misfeature in the FreeBSD (not Linux)
64 # /dev/random, where every superuser write/close causes
65 # an explicit reseed of the yarrow.
66 ) >/dev/urandom

or similar call is made.

Is `/dev/urandom` suitable for simulation purpose?

In the underlying implementation of /dev/urandom is a CSPRNG, the output pool of which has a maximal period of less than 2^(26∗32) − 1, which is then fed into SHA-1 to produce output for /dev/urandom. As such, urandom can obviously produce the amount of random numbers you want, however it can not offer you reproducible results - you will have to cache the sequence you get yourself.

You do not have to worry about what happens when the entropy pool is estimated to be depleted, /dev/urandom will output whatever you request of it. The "theoretical attacks" the urandom(4) man page speaks of are nonexistent. (the "issue" is a huge misunderstanding of what "entropy estimation" is)

Many other PRNGs with large periods exist which reproducible seeding: the Mersenne Twister in C++, xorshift PRNGs, etc. You should be able to adapt any PRNG to the distribution which is suitable for your purposes.

Is /dev/random considered truly random?

The only thing in this universe that can be considered truly is one based on quantum effects. Common example is radioactive decay. For certain atoms you can be sure only about half-life, but you can't be sure which nucleus will break up next.

About /dev/random - it depends on implementation. In Linux it uses as entropy sources:

The Linux kernel generates entropy
from keyboard timings, mouse
movements, and IDE timings and makes
the random character data available to
other operating system processes
through the special files /dev/random
and /dev/urandom.

Wiki

It means that it is better than algorithmic random generators, but it is not perfect as well. The entropy may not be distributed randomly and can be biased.

This was philosophy. Practice is that on Linux /dev/random is random enough for vast majority of tasks.

There are implementations of random generators that have more entropy sources, including noise on audio inputs, CPU temperature sensors etc. Anyway they are not true.

There is interesting site where you can get Genuine random numbers, generated by radioactive decay.

Do I have to reopen /dev/urandom every read?

/dev/urandom is an interface to the kernel's entropy-driven random number generator. It has a manual page, which you can find online and probably also locally. Opening it via fopen() has no documented effect on the data that subsequently can be read. There is no documented limitation to a single read per open, nor would such a limit be plausible.

So I thought: why not call fopen("/dev/urandom", "rb") only once at
the start of the server and then call fread as I need?

You may do so. The main cost is that you permanently devote one of your process's limited number of open files to this purpose, but that might very well not be a problem for you.

The examples I found in the web usually shows fread immediately preceded by fopen. But isn't that expensive?

Opening a file is comparatively expensive, especially with respect to already having it open and therefore doing nothing, but whether it is costly enough to matter depends on how frequently it is done, what else the program may be doing at or around the same time, and other factors. I'm inclined to suppose that none of that is really a consideration for your web examples, however, for I judge that they are presented as they are at least partially for didactic purposes. If you're writing an example then it's a lot easier and clearer to show the fopen() than to describe in words that the FILE you're using is connected to /dev/urandom.

linux's /dev/urandom forward prediction

In principle, yes you can predict. When there is no entropy available dev/urandom becomes a PRNG and its output can in principle be predicted once its internal state is known. In practice it is not that simple, because the internal state is reasonably large and the hash function prevents us working backwards from the output. It can be determined by trial and error, but that is likely to take a long time.

Working ONLY with /dev/random in Java

This answer assumes you know what you're doing. In other cases, use of /dev/random should be minimized.

/dev/random acts like a normal file, therefore any program which can read any file can read from /dev/random. You probably know that cat /dev/random outputs random data directly from it, and if it does this fast enough, you might in fact want to use it. So, if everything else fails, you'll be always be able to directly read that file...

So, if you look at the source of SecureRandom, you discover that it uses SecureRandomSpi for the actual work. It turns out that NativePRNG.Blocking does what you want:

A NativePRNG-like class that uses /dev/random for both seed and random material. Note that it does not respect the egd properties, since we have no way of knowing what those qualities are. This is very similar to the outer NativePRNG class, minimizing any breakage to the serialization of the existing implementation.
Since:
1.8

The problem might be the Since 1.8, which leaves you with the possibility to backport it to earlier platforms, if you can't use Java 8 yet. The sourcecode is available after all.

So, now let's put this in code:

We have to select the specific implementation to use. To find the exact name, we output all available services with the following line:

for (Provider p: Security.getProviders()) p.getServices().forEach(System.out::println);

We then search for Native in there, and we find the folllowing entry:

SUN: SecureRandom.NativePRNGBlocking -> sun.security.provider.NativePRNG$Blocking

This means we can instantiate the SecureRandom object like below to do what you want:

SecureRandom sr = SecureRandom.getInstance("NativePRNGBlocking", "SUN");

A simple test

byte[] b = new byte[10000];
sr.nextBytes(b);
System.out.println(Arrays.toString(b));

takes ages, I had to lower the amount of read bytes. If it works for you, congratulations, you're reading from /dev/random!

Notice though that this class is in the sun.security.provider package, which is not guaranteed to be available everywhere. For example, it will probably not work on Android. If this fine, then this solution will work, otherwise you should just directly read it as a file.

Don't read from /dev/random on Android. Please.

Is /dev/random considered truly random?

The only thing in this universe that can be considered truly is one based on quantum effects. Common example is radioactive decay. For certain atoms you can be sure only about half-life, but you can't be sure which nucleus will break up next.

About /dev/random - it depends on implementation. In Linux it uses as entropy sources:

The Linux kernel generates entropy
from keyboard timings, mouse
movements, and IDE timings and makes
the random character data available to
other operating system processes
through the special files /dev/random
and /dev/urandom.

Wiki

It means that it is better than algorithmic random generators, but it is not perfect as well. The entropy may not be distributed randomly and can be biased.

This was philosophy. Practice is that on Linux /dev/random is random enough for vast majority of tasks.

There are implementations of random generators that have more entropy sources, including noise on audio inputs, CPU temperature sensors etc. Anyway they are not true.

There is interesting site where you can get Genuine random numbers, generated by radioactive decay.



Related Topics



Leave a reply



Submit