Bottleneck with File I/O
From HPCBugBase
HPCBugBase Menu
Submit feedback
Overview
Index
- Defect types (defect patterns)
- Specific defects (individual defects that belong to a defect type)
- Instances (code examples)
- Articles (various info)
- Templates
- Show all categories
- Show all pages
Index by Languages
Contents |
[edit] Fault Description
A typical HPC program needs to read information from input files before beginning the main computation and write results into output files. When using a checkpointing technique, intermediate results are also written to files so that computation can be resumed from these points when necessary. Therefore, the performance of file I/O has a great impact on execution time.
When multiple processes access the file or filesystem at the same time, they can cause a performance problem. For example, the initialization code such as follows can appear as a result of naive parallelization.
FILE *fp = fopen(/* input file */);
if (fp != NULL) {
fread(/* read data */);
if (/* relevant data */) {
/* store the data */
}
}
In this case, all processes try to read the entire input file at the same time, which often slows down the access especially if the re are many processors.
[edit] Statistics (Frequency)
[edit] Other Findings and Contexts
|
Pages referring to this entry: I/O Hotspots |
