Using the Same Randomization Seed in All Processes

From HPCBugBase

Contents 1 Fault Description 2 Statistics (Frequency) 3 Other Findings and Contexts

[edit] Fault Description

HPC applications involving random simulation make use of a pseudo-random number generator. Some pseudo-random number libraries require an explicit initilization with a 'seed' which determines the actual sequence to be generated. For example, srand() must be called to initialize the traditional UNIX function rand(). (Note: it is known that the random number sequence created by rand() is not suitable for serious simulation. The example shown here uses rand() just to illustrate the idea.)

For example, suppose we have the following sequential program. This is a code fragment for approximating the pi value. (Note: using a time-varying seed value is a typical technique to obtain a different random sequence every time you run the program.)

srand(time(NULL));
for (i=0; i<n; i++) {
    double x = rand() / (double)RAND_MAX;
    double y = rand() / (double)RAND_MAX;
    if (x*x + y*y < 1) ++k;
}
return k/(double)n; // approximation of pi

Suppose we turned it into a parallel program in MPI as follows.

srand(time(NULL));
for (i=0; i<n; i+=size) {
    double x = rand() / (double)RAND_MAX;
    double y = rand() / (double)RAND_MAX;
    if (x*x + y*y < 1) ++k;
}
status = MPI_Reduce( ... MPI_SUM... );
...
return k/(double)n;

In this MPI program, it is likely that all processes end up using the same random sequence, as the value of the current time in seconds is expected to be identical. As a result, all processes end up with doing the same computation. A simple way to fix this would be to use a different seed for each process:

srand(time(NULL) + rank);

[edit] Statistics (Frequency)

[edit] Other Findings and Contexts

In the particular example describe above, the defect causes a loss of accuracy with the results. Since the output is an approximation, however, it may not be easy to identify the defect by examining the output.