These hints are to help you to get the most out the HPC resources that are available to you. Although the suggestions are targetting the VPP and some examples are in terms of the VPP, the ideas are generally applicable. If you have any queries or need any help with any of these issues please let us know.

IO is often taken for granted with little thought given to its possible adverse effect on your application (and the system!) or to "tuning" it. However the constant shortage of disk space on all systems and the very poor performance of some applications on the VPP because of IO are pretty good reasons for at least understanding the possible problems thoughtless IO causes.

Basic IO concepts:

"Records", "buffers", "unformatted", ... what do they all mean?

Data formats:

Binary formatted data may be unreadable to you but is a computers "native tongue". Using binary data files has the advantages of much faster IO (less elapsed runtime), less CPU use and generally much smaller files.

File systems and file sizes:

The way filesystems are structured can have a huge negative or positive effect on file sizes and IO performance.

Other IO hints:

Multidimensional array and Fortran buffer considerations.

The typical jobs of some projects will take more cpu-time than allowed for a single run in the batch queues. Some methods used in concatenating several jobs together to complete a large run are:

• checkpointing
• signal catching and handling and
• self-resubmitting batch scripts.

Your approach to managing long running jobs could be much more sophisticated than these suggestions. For example, it is possible to automate a lot of the file manipulation etc. associated with your jobs by straightforward use of the scripts you use to submit your jobs. Whatever you do, it is always advisable to test your job-self-submission and checkpointing procedures on a small job in the express queue before trying it on a full production job - there is nothing worse than biding your time in a batch queue only to find that your script or restart method is buggy!

Developing a large code is a challenging intellectual pursuit and organizational task and it is not possible to offer definitive advice on how to attack such an undertaking. However, we believe the makefiles are a very useful tool in expediting the development cycle.