National Facility users can now access the new SGI Altix system. Please note that the new system is still under development and the service may be less reliable than we would like. Therefore: - there will initially be no charge against your MAS or other grants for usage of the ac. - the sc will continue to operate as our production service for at least 2 weeks. NOTE the important statement about Your Files below. - please read the rest of this newsletter before logging onto the ac. - please be patient if there are problems, and let us know of any problems at help@nf.apac.edu.au We also recommend that you use this time to re-establish the performance and resources required for your programs. In particular the scaling of performance of your parallel jobs should be rechecked. Please keep us informed of both good results and concerns. The ac system - ac.apac.edu.au ------------------------------ The system currently has 8 cpus for interactive work and 640 cpus for computation (512p of these are currently undergoing acceptance). It will be upgraded to 1680 cpus soon. The main web pages to check for information on the new system are: http://nf.apac.edu.au/facilities/userguide/ http://nf.apac.edu.au/facilities/ac/hardware.php http://nf.apac.edu.au/facilities/software/ http://nf.apac.edu.au/facilities/scope/ The user environment is very similar to that on the sc. The main differences you will find is the compilers and the items listed below. Passwords --------- Your password on the ac will be the same as it is on the lc. If you have forgotten your lc password, contact us. Your Files - NOTE: IMPORTANT EFFECT ON FILES ON THE SC ---------- All your files in home and short directories were copied from the sc to the ac on SUNDAY 5 June at midnight. It is YOUR RESPONSIBILITY to transfer any subsequently modified or created files on the sc to the ac before the sc is decommissioned. (We will do a full backup of the sc before decommissioning it, but recovery of such files will cause you delays.) Your environment files .login, .cshrc, .profile and .bashrc have been renamed with an appended "_from_sc" and templates put in their place. DO NOT just copy your originals back in place. First read the new dot files and also read the information about the Default Environment and modules below. Software -------- The 'ac' next to a package on the software page indicates we think it is ready to use. 'ac(-)' indicates a package currently being ported. If there are 'ac(-)' or other packages you urgently require, please let us know. Specific information on using the package on the ac can be found on the web in some cases. Default Environment and modules ------------------------------- On the ac there are no /opt/etc/system_cshrc or profile files for setting up your environment. Instead you can use environment modules. (Modules are now also available for use on the lc.) See http://nf.apac.edu.au/facilities/software/modules.php If you do a "module list" you will see what is in your default environment. Some of these (compilers etc) are set in your .profile (bash) or .login(tcsh) so you can change them. We recommend that you load any other modules interactively, or as part of your batch job to avoid unnecessary path conflicts. "module avail" will give you the list of all available environment modules, and "module show xxxx" will give the full details of what "module load xxxx" will do to your environment. Filesystems ----------- The filesystems on the new system are basically the same structure/conditions as the other systems at the NF. That includes home, short and a $PBS_JOBFS area (available using the jobfs resource request flag). There has been an increase in default /home and /short disk quotas. The limiting of jobfs for large parallel jobs has changed slightly, the limit is now on the jobfs use summed over all nodes of a job. Queues and Queue limits ----------------------- The queue names are the same as on the sc but some default limits have been increased. Use the nf_limits command to view the limits applicable to you. MPI --- The MPI is SGI's proprietary Message Passing Toolkit (MPT) that takes advantage of the high performance NUMAlink interconnect. The MPI job launch commands are locally customised but should be compatible, in behaviour and options, with both SGI/MPICH mpirun and the prun command familiar to SC users. The following are examples: mpirun ./prog mpirun -np $PBS_NCPUS ./prog prun ./prog prun -n $PBS_NCPUS ./prog See the common mpirun/anumpirun/prun man page for more details and the User Guide for an introduction to using MPI on AC. For those using SGI systems at other sites and are using the "dplace" command, contact us for advice on use. Some advanced mpirun or prun option usage may need modification as well. -------------------------------------------------------------------------- ANU Supercomputer Facility help@nf.apac.edu.au and APAC National Facility Phone: +61 2 6125 3437 Leonard Huxley Bldg (No. 56) Fax: +61 2 6125 8199 Australian National University Canberra, ACT, 0200, Australia --------------------------------------------------------------------------