ANU Supercomputer Facility - Annual Report 1998


Collaboration and Outreach


External Use Scheme for Australian Universities

Following its acquisition of the first university-based supercomputer in Australia in 1987, the University undertook to provide ten per cent (now fifteen) of the capacity of the vector supercomputers free of charge to researchers at other Australian universities. Over the ensuing decade, the External Use Scheme has proven very successful. In 1997 it was extended to provide access to the Silicon Graphics PowerChallenge system in accord with the policy of the Australian Cooperative Supercomputer Facility.

Until 1993, the ANU did not involve itself in deciding the relative merits of projects from external universities. Over the first five years of this External Use Scheme, a formula provided by the Australian Vice-Chancellors' Committee was used to allocate time in bulk to each participating university. By the end of 1992, it was clear that the system needed modification. The number of projects grew to be too large, with the result that each project received an inadequate grant. Therefore, after discussions with a number of external coordinators and users, the scheme was altered so that the Supercomputer Time Allocation Committee accepts proposals from external researchers for a reserved ten percent of the computer time available separate from ANU allocations. In considering proposals, the Committee consults national and international referees and is guided by the same principles of academic merit and suitability of the resources to the proposed project as it uses for internal proposals.

Details of external usage of the ANU supercomputers and mass data store facilities is included in the Time Allocation Committee Section of this report. Detailed descriptions of projects can be found in Appendix B.

Australian Cooperative Supercomputer Facility (ACSF)

The ACSF is a collaborative venture initiated by the ANU in 1991 in conjunction with regional computational consortia representing almost all the New South Wales and South Australian universities. The University of Queensland has been a member since 1997. The ACSF operates with the support of several substantial Australian Research Council infrastructure grants and was successful in obtaining additional RIEFP funding for 1999 from the ARC in 1998 for a mass data storage initiative.

The ACSF members operate Silicon Graphics PowerChallenge systems with installations at the ANU, NSW (Australian Technology Park), the University of Adelaide and a large Origin system at the University of Queensland and are pursuing similar activities in visualization and virtual environments. In addition, ACSF members share a Connection Machine CM5 operated by the South Australian Centre for Parallel Computing.

The ACSF has undertaken to implement an external use scheme for researchers from other universities requiring access (see previous section). Researchers at universities within the ACT have access to the Silicon Graphics PowerChallenge system at the ANU as an ACT regional node, ie. their access is on the same basis as ANU users and is not confined to the ten percent reserved for other external universities. Researchers requiring access to the SGI-PowerChallenge from universities in regions which have another ACSF node (SA, NSW and QLD) are generally referred to their regional node, although access is provided for those requiring software or other environmental conditions only available at the ANU.

Dr Gingold attended a number of ACSF board meetings. Facility staff have also been active in the SGI/Cray HPC User Group which is directly relevant to ACSF, attending two interstate meetings and organizing one at the ANU during the year.

The RASH resource and system control software developed and marketed by the Facility for both the Silicon Graphics and Fujitsu systems is available free of charge to ACSF members. It is currently installed on the NSW and ANU systems.

External Education and Training Activities

In previous years workshops specifically for external postgraduate students have been conducted. While this did not occur in 1998, a number of external users, mostly postgraduate students, attended the training courses listed in this report under Education and Training.

Other Outreach Activities

On 29 April a school group from West Wyalong High School (Years 11 and 12) visited the Facility. As mentioned elsewhere in this report, the WEDGE virtual environment system was displayed at the Powerhouse museum in Sydney in December.

Area 3 - Computatational Chemistry Project with Fujitsu Limited

The Area 3 collaborative software development project with Fujitsu Japan was established in 1989 with the aim of porting and optimizing computational chemistry software for the VP-100. The scope of the project was expanded in 1995 to include of mathematical and climate modelling software (in conjunction with DCS and CRES) as well as the preparation of several scientific videos. In 1996 the project was refocussed to concentrate on the porting, optimisation and development of computational chemistry software for Fujitsu computers and a joint application research project with Dr J Gready (Computational Molecular Biology and Drug Design Group, JCSMR) was initiated.

Work undertaken by the project spans both the vector parallel (VPP) and scalar parallel (AP) range of Fujitsu supercomputers. The core software packages of interest to the project in 1998 were the Amsterdam Density Functional, GAUSSIAN 98, and GAMESS-US quantum chemistry codes, the MOPAC and MNDO94 semiempirical codes, and the GROMOS96, AMBER and X-PLOR molecular dynamics/mechanics codes. These packages running on Fujitsu supercomputers form the basis for the application research project.

The joint project with JCSMR is currently studying the enzyme Dihydrofolate Reductase (DHFR). This enzyme is important in processes that construct cellular DNA. Inhibition of the mechanism by several classes of cytotoxic drugs has been explored for treating diseases such as cancer. DHFR catalyses the reduction of a folate substrate to dihydrofolate and then to tetrahydrofolate in the presence of an NADPH cofactor. In 1998 the work has involved studying the polarising effect of the enzyme active site on the substrate and cofactor. This work has involved performing large Hartree-Fock, density functional and second-order perturbation theory calculations on systems containing up to 124 atoms and close to 2000 basis functions.

In 1998 Mr Stephen Titmuss joined the application research project as a Fujitsu sponsored Australian Postgraduate Award Industry (APAI) PhD student. His work aims to investigate the use of MOZYME (a linear scaling version of MOPAC that has been developed by Dr J.J.P Stewart with the direct support of Fujitsu) for the study of enzyme reaction mechanisms, and specifically to contrast the use of MOZYME with existing hybrid quantum chemical/molecular mechanical (QM/MM) methods. Initial work has looked at assessing the effect of changing the various cutoffs employed in MOZYME for calculations on enzyme systems, studying the use of MOZYME for geometry optimisations, and identifying suitable systems for which a comparison with QM/MM is possible.

Overall the project is led by Dr Rendell (ANUSF), who is partially supported by Area 3 funds. Work undertaken in JCSMR is directed by Dr Jill Gready. The project supports 3 other full-time staff working in ANUSF, and a postdoctoral fellow and APAI PhD student located in JCSMR. In 1998 project funds were also used to enable Professor Peter Kollman to attend and talk at the Fourth Australian Molecular Modelling Workshop held in Sydney in July.

During 1998 there were many contacts between the ANU and Fujitsu Japan, including two project planning meetings attended by Dr Rendell in Japan. As well as direct contact with Fujitsu Japan a close collaboration exists between the Area 3 work being performed at the ANU and complimentary work undertaken by the group lead by Dr Nobes, a former ANUSF staff member, at the Fujitsu European Centre for Information Technology (FECIT) in the UK. In particular in November 1998 Drs Rendell and Nobes jointly presented workshops on Gaussian 98 on Fujitsu systems at sites in France, Spain, and Germany.


Area 3 - Written Publications

S.R. Gadre, S.S. Pundlik, A.C. Limaye and A.P. Rendell, Electrostatic investigation of metal cation binding to DNA bases and base pairs, Chemical Communications, (1998) 573.

A. Rendell, A. Bliznyuk, T. Huber, A. Limaye, J. Gready, S. Greatbanks and R. Schmidt, Hybrid quantum mechanical/molecular mechanical studies (QM/MM) of enzyme reaction mechanisms: Building the computational engine, Proceedings of the High Performance Computing Conference, HPC Asia 1998 Singapore, ISBN 981-04-0634-7.
T. Huber and A.E. Torda, Protein fold recognition without Boltzmann statistics or explicit physical basis, Prot. Sci. 7 (1998) 1-8.

T. Huber and W.F. van Gunsteren, SWARM-MD: searching conformational space by cooperative molecular dynamics, J. Phys. Chem. A, 102 (1998) 5937-5943.

A.A. Bliznyuk and A.P. Rendell, Faster semiempirical gradients, J. Comp. Chem. (in press).

D. Ayers, T. Huber and A.E. Torda, Protein fold recognition force fields: unusual construction strategies, Proteins (in Press) .

S.P. Greatbanks, J.E. Gready, A.C. Limaye and A.P. Rendell, Enzyme polarization of substrates and cofactor of dihydrofolate reductase by different theoretical methods, Proteins (submitted).

Area 3 - Oral and poster presentations

A. Rendell, Quantum mechanical studies of enzyme reaction mechanisms: building the computational engine, invited talk, XII International Conference on Computers in Chemical Research and Education, Uni. of Pune, India, Jan 5-9, 1998.

A.C. Limaye and A. Rendell Performance and tuning of Gaussian 94 for Fujitsu parallel supercomputers, poster, XII International Conference on Computers in Chemical Research and Education, Uni. of Pune, India, Jan 5-9, 1998.

T. Huber, Protein structure guesstimation by fold recognition and hierarchical optimisation, contributed oral presentation, the Fourth Australian Molecular Modelling Workshop, Sydney, 19th - 22nd July 1998.

A.A. Bliznyuk and A.P. Rendell, Faster semiempirical gradients, poster presentation, the Fourth Australian Molecular Modelling Workshop, Sydney, 19th - 22nd July 1998.

S.P. Greatbanks, J.E. Gready, A.P. Rendell, and A.C.Limaye, An ab initio study of polarisation in the reaction mechanism of the enzyme dihydrofolate reductase, poster presentation, the Fourth Australian Molecular Modelling Workshop, Sydney, 19th - 22nd July 1998.

A.J. Limaye, A.P. Rendell, A.A. Bliznyuk, T. Huber, J. Gready and S. Greatbanks, Hybrid quantum mechanical/molecular mechnaical studies (QM/MM) of enzyme reaction mechanisms: Building the computational engine, poster presentation, the Fourth Australian Molecular Modelling Workshop, Sydney, 19th - 22nd July 1998.

S. Titmuss, J.E. Gready, A.P. Rendell and A.A. Bliznyuk, Evaluation of the potential of MOZYME for the study of enzyme reaction mechanisms, poster presentation, the Fourth Australian Molecular Modelling Workshop, Sydney, 19th - 22nd July 1998.

S.P. Greatbanks, J.E Gready, A.C. Limaye and A.P.Rendell, An ab initio study of polarisation in the reaction mechanism of the enzyme dihydrofolate reductase, poster presentation, 34th Symposium for Theoretical Chemistry (Large Molecules), 20-24 September 1998, Gwatt, Switzerland.

A.A. Bliznyuk and A.P. Rendell, Faster gradients in semiempirical methods, poster presentation, 34th Symposium for Theoretical Chemistry (Large Molecules), 20-24 September 1998, Gwatt, Switzerland.

A.A. Bliznyuk, Semiempirical life at ANUSF, seminar, Fujitsu European Centre for Information Technology, 17th September 1998.

A. Rendell, Tuning, running and using Gaussian 98 on Fujitsu parallel supercomputers, A series of talks presented during November 1998 at the Fujitsu International Supercomputer User group Meeting (Toulouse, France), the Centro de Supercomputacion de Galicia (Santiago de Compostela, Spain), and the Gaussian 98 workshop (Wuerzburg, Germany).

T. Huber, Protein structure prediction: The refined and the uncouth, invited seminar, Department of Pharmaceutical Chemistry, University California San Francisco, December 10, 1998.

T. Huber, Threading score functions without Boltzmann statistics, Third meeting on the critical assessment of techniques for protein structure prediction, Asilomar, California, December 13-17 1998.

Area 4 - Parallel Mathematical Subroutine Library Project with Fujitsu Limited

The Area 4 project commenced in 1992 and involves research and development of mathematical library algorithms and code for Fujitsu's VPP300/700 supercomputers. The Supercomputer Facility manages this project and three of the staff members are actively involved in the work. Dr M Kahn is responsible for day-to-day management of the project and the coordination of planning and activities across campus. Academic direction and leadership of the project is given by Professor R Brent, CSL, RSISE and Professor M. Osborne, Program in Advanced Computation, CMA, SMS. Professor Brent took up a position as Professor of Computing Science at Oxford University but remains involved in the Area 4 project. Staff and students working on the project are located in SMS, Computer Science Laboratory, RSISE and ANUSF.

Three research fellow positions are funded under this project. Throughout 1998 these positions were held by Dr David Harrar II, Dr Lutz Grosz and Dr Zhou Bing Bing. Dr Markus Hegland, CSL, RSISE continued his role in the project, in particular, as supervisor to Mr Geoff Keating who holds an Australian Industry Postgraduate Award scholarship to work towards a PhD in the Computer Science Laboratory, RSISE. This scholarship is supported by Fujitsu.

Within ANUSF Mr Gavin Mercer worked half-time as software engineer and program librarian for part of 1998 whilst Dr M Dow and Dr M Kahn were involved in algorithm and code development. Dr Dow, who is employed half-time as an Academic Consultant, was involved in modifying some routines and Dr Kahn worked extended hours on top of a substantive half-time position to manage the project and contribute to code development.

The release of Version 14 L10 of the Fujitsu SSLIIVP library during the year has brought many more of the Area 4 developed routines into commercial usage. The performance of these routines, both for one-processor and multi-processor codes, exceeds that of other mathematical subroutine libraries on the VPP300.

Area 4 - Publications

R. Brent, L. Grosz, D., II Harrar, M. Hegland, M. Kahn, G. Keating, G. Mercer, O. Nielsen, M. Osborne, B. Zhou, M. Nakanishi, Development of a Mathematical Subroutine Library for Fujitsu Vector Parallel Processors, ICS '98. Conference proceedings of the 1998 International Conference on Supercomputing, pp.13-20, ACM July 1998.

R. Brent, L. Grosz, D. Harrar II, M. Hegland, M. Kahn, G. Keating, G. Mercer, M. Osborne, B. Zhou, Design of the Scientific Subroutine Library for the Fujitsu VPP300, in K.Y. Lam, B.C.Khoo, K. Kumar: Proceedings of the 3rd High Performance Computing Asia, Singapore, September 1998, pp 424-438.

L. Grosz, On the Implementation of Algebraic Multi-level Methods on Vector and Parallel-Vector Computers, in S. Oliveria, D. Stewart: Proceedings of the Workshop on New Methods in Applied and Computational Mathematics 1998, Hervey Bay. Australian National University, Centre for Mathematics and its Applications, to appear.

D.L. Harrar II and M.R. Osborne, Solving large-scale eigenvalue problems on vector parallel processors, ed Palma, J. et al, in Vector and Parallel Processing, VECPAR'98, Lecture Notes in Computer Science, 1999, to appear.

D.L. Harrar II and M.R. Osborne, Parallel solution of some large-scale eigenvalue problems arising in chemistry and physics, ed Kagstrom, B. et al, in Applied Parallel Computing, PARA'98, Lecture Notes in Computer Science, No. 1541, pp 216-223, 1998.

M. Hegland, M. Kahn and M. Osborne, A parallel algorithm for the reduction to tridiagonal form for eigendecomposition, accepted for publication, Siam Journal on Scientific Computing.

M. Hegland and M. Osborne, 1998, Algorithms for Block Bidiagonal Systems on Vector and Parallel Computers, ICS '98. Conference proceedings of the 1998 international conference on Supercomputing, pages 1-6, ACM 1998.

Special Services to Other Internal and External Organizations

Dr Gingold served on the Duffield Scholarship Committee at MSSSO and is Associate Director of the Computational Techniques and Applications Conference CTAC99 to be held at the ANU in 1999. He is also on the Executive Committee of the Fujitsu International Supercomputer Users Group. Dr Gingold also served in 1998 on the 'Expert Group' established to advise the Interim Board reporting to DEETYA on a Centre of Expertise in High Performance Computing and Communications.

ANU Supercomputer Facility - Home Page | Contact us
The Australian National University