More IO hints



Multidimensional arrays

As with all array-based computing, try to avoid strided memory access through having the "wrong" inner loop index. The linear layout of array elements in memory and the banked nature of memory means that it is easy to degrade IO and compute performance by having loops interchanged.

Fortran arrays:
the leftmost array index varies fastest in memory (i.e. a 2D array has element order a(1,1), a(2,1), a(3,1), ...., a(1,2), a(2,2), ... in memory). So for any nested loop structure, the index of the inner DO loop should use the leftmost array index. For IO, use something like:
       do j=1,ny
          write(8) (c(i,j), i=3,nx-2)
       enddo
C arrays:
the rightmost array index varies fastest in memory (i.e. a 2D array has element order a[0][0], a[0][1], a[0][2], ...., a[1][0], a[1][1], ... in memory). So for any nested loop structure, the index of the inner loop should use the righttmost array index. For IO, use something like:
       for(i=0;i<nx;i++)
          write(DATA, &c[i][0], 8*ny);
       
Fortran buffer size

When using sequential IO to a buffered filesystems from Fortran on the VPP, it is a simple matter to artificially increase the size of your IO records. The default buffer size is reasonably small (4 or 8kB) which leads to slow standard NFS IO being used from the secondary PEs. The -Wl,-g runtime option can be used to increase the buffersize - it specifies the sequential IO buffer size (in kBs) for your program to use, eg.
a.out -Wl,-g128
causes writes to be buffered to 128kB doing a single write to disk. Please use this option as much as possible.