Random Number Generators and Information Security

From Computing and Software Wiki

Revision as of 04:01, 10 December 2007 by Caoff (Talk)

(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)

A random number generator (often abbreviated as RNG) is a computational or physical device designed to generate a sequence of numbers or symbols that lack any pattern, i.e. appear random. Computer-based systems for random number generation are widely used, but often fall short of this goal, though they may meet some statistical tests for randomness intended to ensure that they do not have any easily discernible patterns. Methods for generating random results have existed since ancient times, including dice, coin flipping, the shuffling of playing cards, the use of yarrow stalks in the I Ching, and many other techniques.
The many applications of randomness have led to many different methods for generating random data. These methods may vary as to how unpredictable or statistically random they are, and how quickly they can generate random numbers.
Before the advent of computational random number generators, generating large amount of sufficiently random numbers (important in statistics) required a lot of work. Results would sometimes be collected and distributed as random number tables.^[2]
^[5]

Relevance to Information Security

Random numbers, or pseudo-random numbers, are used in many aspects of computer science. This wiki page discusses the particular relevance of random numbers with cryptography.
For example, the Needham-Schroeder Protocol depends on a good random number generator for truly unpredictable numbers, which are used to protect messages from attackers. A random number generator that gives predictable output would render the protocol entirely meaningless. Since a system is only as secure as its weakest component, random number generators are a crucial part of information security.
^[1]

Vulnerability to attack

Quality in the random number generation (RNG) process is almost always required for security, and lack of quality generally provides attack vulnerabilities and so to lack of security, even to complete compromise, in cryptographic systems. The RNG process is particularly attractive to attackers because it is typically a single isolated hardware or software component easy to locate. If the attacker can substitute pseudo-random bits generated in a way he can predict, security is totally compromised, yet generally undetectable by any upstream test of the bits. Furthermore, such attacks require only a single access to the system that is being compromised. No data need be sent back in contrast to, say, a computer virus that steals keys and then e-mails them to some drop point.^[3]

Examples of inadequate random number generators

Early versions of Netscape's Secure Socket Layer (SSL) encryption protocol used pseudo-random quantities derived from a PRNG seeded with just three values, the time of day, the process ID, and the parent process ID. These quantities are often relatively predictable, making that version of SSL insecure. The problem was discovered in 1995 by Ian Goldberg and David Wagner, who had to reverse engineer the object code because Netscape refused to reveal the details of its random number generation. The SSL RNG was fixed in later releases (version 2 and higher) by more robust seeding.
Microsoft uses an unpublished algorithm to generate random values for its Windows operating system. These random quantities are made availabe to users via the CryptGenRandom utility. In November 2007, Leo Dorrendorf et al. from the Hebrew University of Jerusalem and University of Haifa published a paper titled Cryptanalysis of the Random Number Generator of the Windows Operating System. The paper presented serious weaknesses in the Microsoft approach. The paper's conclusions were based on based on disassembly of the code in Windows 2000, but according to Microsoft apply to XP as well.
The U.S. National Institute of Standards and Technology has published a collection of "deterministic random bit generators" it recommends as NIST Special Publication 800-90. One of the generators, Dual_EC_DRBG, was favored by the National Security Agency. Dual_EC_DRBG uses elliptic curve technology and includes a set of recommended constants. In August 2007, Dan Shumow and Niels Ferguson of Microsoft showed that the constants could be constructed in such a way as to create a secret backdoor to the algorithm.^[3]

Attacks on software random number generators

Just as with other components of a cryptosystem, a software random number generator should be designed to resist certain attacks. Exactly which attacks must be defended against depends on the system, but here are a few:

If an attacker obtains most of the stream of random bits, it should be infeasible for them to compute any additional parts of the stream.
If an attacker observes the internal state of the random number generator, they should not be able to work backwards and deduce previous random values.
If an attacker observes the internal state of the random number generator, they will necessarily be able to predict the output until enough additional entropy is obtained. However, if entropy is added incrementally, the attacker may be able to deduce the values of the random bits that were added and obtain the new internal state of the random number generator (a state compromise extension attack).
If an attacker can control the supposedly random inputs to the generator, they may be able to "flush" all the existing entropy out of the system and put it into a known state.
When a generator starts up, it will often have little or no entropy (especially if the computer has just been booted and followed a very standard sequence of operations), so an attacker may be able to obtain an initial guess at the state.^[3]

Attacks on hardware random number generators

A number of attacks on hardware random number generators are possible, including trying to capture radio-frequency emissions from the computer (obtaining hard drive interrupt times from motor noise, for example), or trying to feed controlled signals into a supposedly random source (such as turning off the lights in a lava lamp or feeding a strong, known signal into a sound card).^[3]

Two kinds of randomness

True randomness - Physical methods

The earliest methods for generating random numbers - dice, coin flipping, roulette wheels are still used today, mainly in games and gambling as they tend to be too slow for applications in statistics and cryptography.
Some physical phenomena, such as thermal noise in zener diodes appear to be truly random and can be used as the basis for hardware random number generators. However, many mechanical phenomena feature asymmetries and systematic biases that make their outcomes not truly random. The many successful attempts to exploit such phenomena by gamblers, especially in roulette and blackjack are testimony to these effects.
There are several imaginative sources of random numbers online. A common technique is hashing a frame of a video stream from an unpredictable source. Most notable perhaps was Lavarand which used images of a number of lava lamps. Lithium Technologies uses a camera pointed at the sky on a windy and cloudy day. Random.org has a more obvious approach of listening to atmospheric noise. Details about how they turn their input into random numbers can be found on their respective sites.^[2]

Completely randomized design falls within the category of true random number generation. The generation of true random numbers outside the computer environment is based on the theory of entropy. Sources of entropy include nuclear decay and atmospheric conditions. HotBits uses radioactive decay, while Random.org uses radio noise to generate randomness.^[2]

HotBits is an Internet resource that brings genuine random numbers, generated by a process fundamentally governed by the inherent uncertainty in the quantum mechanical laws of nature, directly to your computer in a variety of forms. HotBits are generated by timing successive pairs of radioactive decays detected by a Geiger-Müller tube interfaced to a computer. You order up your serving of HotBits by filling out a request form specifying how many random bytes you want and in which format you'd like them delivered. Your request is relayed to the HotBits server, which flashes the random bytes back to you over the Web. Since the HotBits generation hardware produces data at a modest rate (about 100 bytes per second), requests are filled from an “inventory” of pre-built HotBits. Once the random bytes are delivered to you, they are immediately discarded—the same data will never be sent to any other user and no records are kept of the data at this or any other site.^[6]

Pseudo-randomness - Computational methods

Computational methods use mathematical formulae or simply precalculated tables to produce sequences of numbers that appear random. Since the computer follows a deterministic algorithm to generate these numbers, they are inherently predictable.

Middle-squared Method, 1946
Blum Blum Shug, 1986
CryptGenRandom

Verdict

Physical Methods are:

Slower than computational methods, since sequences of random numbers are generated in real-time through measuring physical phenomena.
Nondeterministic.
Truly unpredictable; this characteristic "can be traced to the laws of quantum mechanics^[2]", which are unpredictable as we know them.

Computational Methods:

Efficient: long sequences of numbers can be produced very quickly.
Deterministic: given the seed state, any sequence of numbers can be reproduced anytime. This can be an advantage if past sequences are needed at a later date for validation or analysis purposes. However, it may make the system vulnerable to replay attacks (see Vulnerability to Attacks sectiona bove).
Periodicity: all sequences eventually repeat themselves. Despite sounding dubious, this characteristic does not present a practical problem, as the periods of most PRNG's used today are very large and thus sequences do not actually repeat in realistic situations.^[4]

The basic difference between PRNGs and TRNGs can be explained with the analogy of rolling a die. PRNGs produce sequences of numbers using mathematical formulae or precalculated lists (ie. someone performed many trials and recorded the results), and output numbers from the sequences (ie. the next number on the pre-recorded list). The numbers appear random, but they are really predetermined.^[4] If an attacker obtains the algorithm (the list of numbers) and the seed state (the starting number), he or she can predict every "random" number in the sequence.

TRNGs measure physical phenomena (ie. actually rolling the die and reading out the number).^[4] They are as unpredictable as the physical phenomenon used. Even if the attacker knows how the numbers are generated and obtains a record of past numbers, he or she cannot predict the next number with any certainty.

Both methods have their advantages and disadvantages. When efficiency is a priority, such as in simulation and modeling applications, computational methods are more appropriate. When unpredictability is crucial, such as in the case of data encryption, physical generators are generally preferred.

Cryptographically secure pseudo-random number generators

A cryptographically secure pseudo-random number generator (CSPRNG) is a pseudo-random number generator (PRNG) with properties that make it suitable for use in cryptography.
Many aspects of cryptography require random numbers, for example:

Key generation
Nonces
Salts in certain signature schemes, including ECDSA, RSASSA-PSS.
One-time pads

The "quality" of the randomness required for these applications varies. For example creating a nonce in some protocols needs only uniqueness. On the other hand, generation of a master key requires a higher quality, such as more entropy. And in the case of one-time pads, the information-theoretic guarantee of perfect secrecy only holds if the key material is obtained from a true random source with high entropy.
Ideally, the generation of random numbers in CSPRNGs uses entropy obtained from a high quality source, which might be a hardware random number generator or perhaps unpredictable system processes — though unexpected correlations have been found in several such ostensibly independent processes. From an information theoretic point of view, the amount of randomness, the entropy that can be generated is equal to the entropy provided by the system. But sometimes, in practical situations, more random numbers are needed than there is entropy available. Also the processes to extract randomness from a running system are slow in actual practice. In such instances, a CSPRNG can sometimes be used. A CSPRNG can "stretch" the available entropy over more bits.
When all the entropy we have is available before algorithm execution begins, we really have a stream cipher. However some crypto system designs allow for the addition of entropy during execution, in which case it is not a stream cipher equivalent and cannot be used as one. Stream cipher and CSPRNG design is thus closely related.^[7]

Requirements

The requirements of an ordinary PRNG are also satisfied by a cryptographically secure PRNG, but the reverse is not true. CSPRNG requirements fall into two groups: first, that their statistical properties are good (passing statistical randomness tests); and secondly, that they hold up well under serious attack, even when part of their initial or running state becomes available to an attacker.
Every CSPRNG should satisfy the "next-bit test". The next-bit test is as follows: Given the first k bits of a random sequence, there is no polynomial-time algorithm that can predict the (k+1)th bit with probability of success higher than 50%. Andrew Yao proved in 1982 that a generator passing the next-bit test will pass all other polynomial-time statistical tests for randomness.
Every CSPRNG should withstand 'state compromise extensions'. In the event that part or all of its state has been revealed (or guessed correctly), it should be impossible to reconstruct the stream of random numbers prior to the revelation. Additionally, if there is an entropy input while running, it should be infeasible to use knowledge of the input's state to predict future conditions of the CSPRNG state.
Example: If the CSPRNG under consideration produces output by computing bits of π in sequence, starting from some unknown point in the binary expansion, it may well satisfy the next-bit test and thus be statistically random, as π appears to be a random sequence. (This would be guaranteed if pi is a normal number, for example.) However, this algorithm is not cryptographically secure; an attacker who determines which bit of pi (ie the state of the algorithm) is currently in use will be able to calculate all preceding bits as well. Most PRNGs are not suitable for use as CSPRNGs and will fail on both counts. First, while most PRNGs outputs appear random to assorted statistical tests, they do not resist determined reverse engineering. Specialized statistical tests may be found specially tuned to such a PRNG that shows the random numbers not to be truly random. Second, for most PRNGs, when their state has been revealed, all past random numbers can be retrodicted, allowing an attacker to read all past messages, as well as future ones. CSPRNGs are designed explicitly to resist this type of cryptanalysis.^[7]

Conclusion

External Links

RANDOM.ORG
HotBits
Lavarnd
Psychic Science
GraphPad Software
Research Randomizer
The pLab Project

References

1. Bishop, M. 2004. Introduction to Computer Security. Prentice Hall PTR.
2. Wikipedia: Random Number Generation. 2007. Retrieved November 27, 2007.
3. Wikipedia: Random number generator attack. 2007. Retrieved November 27, 2007.
4. Haahr, M. 2007. Random.org. Retrieved November 27, 2007. Random.org
5. California Soil Resource Lab. 2007. Retrieved November 27, 2007.
6. HotBits. 2007. Retrieved November 27, 2007.
7. Wikipedia: Cryptographically secure pseudorandom number generator. 2007. Retrieved November 27, 2007.
8. Lavarnd. 2007. Retrieved November 27, 2007.

--Caoff

Random Number Generators and Information Security

From Computing and Software Wiki

Contents