SOFTWARE / USER GUIDE
High Performances Computing traditionally referred to as HPC is the application of technology to highly complex scientific, engineering, and business analytics workloads. Why we want to use INSPEM’s HPC? INSPEM’s HPC deliver a flexible and scalable choice for the application of technology to highly complex scientific, engineering, and business analytics workloads:-
- Scalable solutions End-to-end architecture Stable Sharable
- Open Standard for interoperability
Uses of HPC technologies in research and education is a desirable step for university to produce top-quality graduates and researcher for our country to enhance the educational process, to encourage young people to pursue careers in science and technology and to support those career choices.
INSPEM'S HPC unit consist of 2 units of server (Sun Fire V1280 using INSPEM1.upm.edu.my dns name and Sun Fire V240r using INSPEM1a.upm.edu.my) and 2 units of consule (Sun Blade 150 using inform.upm.edu.my and INSPEM2.upm.edu.my).
All the machines have been installed with the default Solaris 9 install. All machines have also been updated using the Solaris 9 Maintenance Update 4 (download from http://sunsolve.sun.com, free registration required). This brings up the operating system to the level of Solaris 9 8/03 ie. Solaris 9 Maintenance Update 4 is equivalent to Solaris 9 8/03. Documentation for it is at :
All machines have been updated with the latest recommended Solaris Patch Cluster as of January 13, 2004. See http://sunsolve.sun.com/pub-cgi/show.pl?target=patches/patch-access
Sun HPC Clustertools is Sun's implementation of the MPI standard, optimized for use on Sun Solaris running on Sun UltraSparc hardware. Sun HPC Clustertools is installed in /opt/SUNWhpc on INSPEM 1 and INSPEM 2 only. Installation was done according to the directions stated in the documentation. NFS was used to share the installation directory. Please see http://docs.sun.com for details on installation directions.
Here are the information about the newest Sun HPC ClusterTools 8 Software :
■ Support for the Linux operating system — Sun HPC ClusterTools 8 includes support for Red Hat Linux versions 4 and 5 and SuSe Linux versions 9 and 10.
■ Enhanced scalability — ClusterTools 8 software now supports clusters of 1024 nodes and 4096 processes.
■ MPI profiling with VampirTrace on the Solaris OS — The VampirTrace program tracing tool is integrated into Open MPI version 1.3 and subsequent versions.
■ uDAPL BTL multi-rail support — the cluster contains multiple Infiniband (IB) Host Channel Adapter (HCA) cards or other physical connections into the IB fabric from a single node, Sun HPC ClusterTools 8 allows selection to a particular HCA or physical connection when running using the uDAPL BTL.
■ libstlport4 support — The MPI C++ library can now be compiled and linked against either the C++ standard or the STLPORT4 library.
■ Support for the MPI OpenFabrics communications — Sun HPC ClusterTools 8 supports performance tuning for Infiniband and iWARP communications in Linux.
Related Software
Sun HPC ClusterTools 8 software works with the following versions of related software:
■ Solaris™ 10 11/06 OS, or any subsequent Solaris 10 OS release that supports Sun
HPC ClusterTools 8 software. If you are running Solaris 10 11/06 OS, you must
install patch 125792-01 (for systems with SPARC® processors) or patch 125793-01
(for systems with AMD™ processors), plus any patches that those patches
require.
■ Red Hat Linux versions 4 and 5 (RHEL)
■ SuSe Linux versions 9 and 10 (SLES)
■ Sun™ Studio 10, 11, and 12 C, C++, and Fortran compilers
■ gcc Linux compiler versions 3.3.3, 3.4.6, and 4.1.2
■ Distributed resource management (DRM) frameworks:
■ Sun™ Grid Engine Version 6
■ Altair PBS Professional 9.2 or Cluster Resources Torque 2.3
■ Debugging software:
■ The TotalView 8.4.1 debugger from TotalView Technologies (formerly Etnus) supports debugging MPI applications on SPARC and AMD systems running the Solaris OS.
■ The DDT 2.1.3 debugger from Allinea also supports debugging on SPARC- and AMD-based systems running the Solaris OS.
Installing the ClusterTools 8 Linux Packages
Sun HPC ClusterTools 8 software supports Red Hat Linux (RHEL) versions 4 and 5 and SuSe Linux (SLES) versions 9 and 10. The Linux packages are delivered in RPM format.
For each version of Linux, there are two types of ClusterTools 8 RPMs: one built with Sun Studio compilers, and the other built with the GNU compiler gcc.
Mellanox Host Channel Adapter Support
The Mellanox ConnectX Infiniband Host Channel Adapter (HCA) requires the following software in order to run Sun HPC ClusterTools 8:
■ Solaris™ 10 5/08 OS, or any subsequent Solaris 10 OS release that supports Sun HPC ClusterTools 8 software.
■ Infiniband software, Update 2. This download is available at http://www.sun.com/download/index.jsp?cat= Hardware%20Drivers&tab=3&subcat=InfiniBand
Note – The Infiniband software, Update 2, requires a firmware upgrade to the
HCA.
Change in Open MPI Terminology
Sun HPC ClusterTools 8 is based on Open MPI version1.3. In that version of Open MPI, the Process lauinch Subsystem (PLS), a component of the Open Runtime Environment (ORTE), was renamed to Process Launch Module (PLM). This means that all MCA parameters under the ORTE pls framework were renamed to begin with plm instead.
Sun HPC ClusterTools 8 Software
Sun HPC ClusterTools 8 software requires the Solaris 10 Operating System (Solaris 10 OS). All programs that execute under the Solaris 10 OS will execute in the Sun HPC ClusterTools environment. Sun HPC ClusterTools 8 software supports Sun Studio 10, 11, and 12 C, C++, and Fortran compilers on the Solaris OS, and the Sun Studio 12 and GNU gcc compilers on Linux.
Sun HPC ClusterTools 8 software can run MPI jobs of up to 4096 processes on as many as 1024 nodes. It also provides support for spawning MPI processes. The Sun HPC ClusterTools 8 software runs on clusters connected by any TCP/IP-capable interconnect, such as high-speed Ethernet, Gigabit Ethernet, and Infiniband.
Open Runtime Environment (ORTE)
Sun HPC ClusterTools 8 is based on the Open MPI message-passing interface. Open MPI operates using the Open Runtime Environment (ORTE). ORTE starts jobs and provides some status information.
The Open MPI mpirun and mpiexec commands are actually symbolic links to the orterun command. All three commands perform the same function, which is to launch MPI jobs.
ORTE is compatible with a number of other launchers, including rsh/ssh, Sun Grid Engine, and PBS Professional/Torque.
Each of ORTE’s primary operations is summarized in the sections that follow. Subsequent chapters contain the procedures.
Executing Programs With mpirun
Sun HPC ClusterTools 8 software can start both serial and parallel jobs.
About the mpirun Command :
The mpirun command controls several aspects of program execution in Open MPI. mpirun uses the Open Run-Time Environment (ORTE) to launch jobs.
Syntax for the mpirun Command
The following example shows the general single-process syntax for mpirun:
% mpirun [options] [program-name] |
For a simple SPMD (Single Process, Multiple Data) job, the typical syntax is:
% mpirun -np x program-name |
For jobs involving multiple instructions, the command syntax appears similar to the following:
% mpirun [options] [program-name] : [options2] [program-name2] ... |
For an MPMD (Multiple Program, Multiple Data) parallel application, the syntax follows this form:
% mpirun -np x program1 : -np y program2 |
About the mpirun Command
The mpirun command controls several aspects of program execution in Open MPI. mpirun uses the Open Run-Time Environment (ORTE) to launch jobs. If you are running under distributed resource manager software, such as Sun Grid Engine or PBS, ORTE launches the resource manager for you. If you are using rsh/ssh instead of a resource manager, you must use a hostfile or host list to identify the hosts on which the program will be run. When you issue the mpirun command, you specify the name of the hostfile or host list on the command line; otherwise, mpirun executes all the copies of the program on the local host, in round-robin sequence by CPU slot.
The following example shows the general single-process syntax for mpirun:
% mpirun [options] [program-name]
|
For a simple SPMD (Single Process, Multiple Data) job, the typical syntax is:
% mpirun -np x program-name |
For jobs involving multiple instructions, the command syntax appears similar to the following:
% mpirun [options] [program-name] : [options2] [program-name2] ... % mpirun -np x program-name |
For an MPMD (Multiple Program, Multiple Data) parallel application, the syntax follows this form:
% mpirun -np x program1 : -np y program2 |
This command starts x number of copies of the program program1, and then starts y copies of the program program2.
Sun HPC ClusterTools software provides middleware to facilitate and manage a workload of highly resource-intensive applications on Sun symmetric-multiprocessor (SMP) systems, as well as clusters of these systems. Additionally, it provides the software development environment for analyzing, debugging, and performance monitoring of Message Passing Interface (MPI) applications on Sun systems and clusters of systems.
-
Sun MPI application delivers thread-safe message passing designed for communication among multiple nodes in clusters as well as among processes on the same symmetric multiprocessor (SMP). The Sun MPI application is compliant with the new MPI 2 standard including full functionality such as one-sided communications, MPI I/O, and name publishing.
-
Sun MPI I/O maximizes parallel I/O capabilities for message-passing among multiple nodes in clusters. The Sun MPI application delivers thread-safe message passing optimized for Sun symmetric multiprocessors (SMP) and clusters thereof.
-
Prism graphical programming environment allows developers to execute, debug, and tune Sun MPI applications. In addition, it provides a number of data and program visualization capabilities.
-
Sun Scalable Scientific Subroutine Library (S3L) provides scalable parallel functions and tools for scientific applications.
-
CRE (Cluster Runtime Environment) provides basic job-launching and load-balancing capabilities. The Sun Cluster Runtime Environment supports MPI jobs of up to 2048 processes, running across up to 256 compute nodes.
-
Cluster Console Manager (CCM) enables administrators to open windows to each node in a cluster and to initiate operations either across all nodes or across subsets of nodes
Please see http://docs.sun.com/app/docs/coll/hpc-clustertools8?l=en for the Sun HPC ClusterTools 8 documentation collection.
Sun Forte Compilers are Sun's optimized compilers for use on Sun's UltraSparc hardware. The installed version is Sun Forte 7 and was installed by ABS engineers.
The compilers were installed in /opt/SUNWspro. Users will need to have /opt/SUNWspro/bin in their path to use the Sun Forte Compilers. The default user settings already have the directories listed above in the path and INSPEM does not have to do any additional updates to take advantage of the tools at this point. Some basic commands include:
cc Sun C Compiler
CC Sun C++ Compiler
f77/f90/f95 Sun Fortran 77/90/95 Compiler
The following website is the complete documentation for the Sun Forte 7 Compilers. http://docs.sun.com/source/816-2460/Preface.html
Sun ONE Grid Engine, also referred to as SGE is Sun's open source queue management system. SGE is used in a HPC cluster to manage the available computing resources. SGE is installed in /opt/sge on the INSPEM systems. Installation was done according to the directions stated in the documentation. NFS was used to share the installation directory. Please see http://docs.sun.com for details on installation directions.
Users will need to execute /opt/sge/default/common/settings.sh (or settings.csh depending on your shell) to use SGE. The default user settings already have the directories listed above in the path and INSPEM does not have to do any additional updates to take advantage of the tools at this point. Some basic commands include:
qsub <script> submit <script> to the default job queue
qstat –f check the status of all the job queues
qmon the X-Windows based GUI for SGE
The Sun ONE Grid Engine manual is available online at: http://docs.sun.com/app/docs/doc/817-3606/abbaa?l=en&a=view&q=Sun+ONE+Grid+Engine+manual
The Sun Software Companion tools are open source applications including many GNU utilities that have been compiled by Sun engineers. These tools are packaged as the Solaris Software Companion and the ISO image is available for download at http://www.sun.com/.
The tools are installed in /opt/sfw and users should have /opt/sfw/bin in their path to use these tools. Most of these tools are command line based eg. gcc and gdb. The default user settings already have the directories listed above in the path and INSPEM does not have to do any additional updates to take advantage of the tools at this point. See http://dlc.sun.com/pdf/806-5202/806-5202.pdffor details.
Whatever the objective of your work—an algorithm, analysis, graph,report, or software simulation—MATLAB gets you there. The flexible, interactive MATLAB language lets technical professionals like you express problems in engineering and science simply. Extensive and powerful numeric computing methods and graphics let you test your ideas and explore alternatives easily, while the integrated development environment lets you produce fast, practical results. Matlab is located in /export/home/matlab/bin/matlab/ .Please make sure the matlab server is running before the the user want to use.
The MATLAB language is designed for interactive or automated computation. Matrix-optimized functions let you perform interactive analyses, while the structured language features let you develop your own algorith ms and applications. The versatile language lets you tackle a range of tasks, including data acquisition, analysis, algorithm development, system simulation, and application development. Language features include data structures, objectoriented programming, graphical user interface (GUI) development tools, debugging features, and the ability to link in your C, C++, Fortran, and Java routines. With more than 600 mathematical, statistical, and engineering functions,MATLAB gives you immediate access to high-performance numeric computing. The numerical routines are fast, accurate, and reliable. These algorithms, developed by experts in mathematics, are the foundation of the MATLAB language. For more detail, please goto http://www.mathworks.com/
S-PLUS is the premier solution for exploratory data analysis and statistical modeling. With over 4,200 data analysis functions, including the most comprehensive set of robust and modern methods available anywhere, S-PLUS allows you to perform more insightful analyses, create revealing graphics and make more informed business decisions.
With point-and-click ease, you can import your data, select your statistical functions and display your results. Powerful techniques are at your fingertips. When your analysis requires a new method or approach, you can modify existing methods of develop new ones with the award-winning S programming language. With S-PLUS Professional, it is easy to examine and visually explore data, run functions one step at a time, and visually compare models for fit.
S-PLUS provides a popular platform for business intelligence and decision support applications that leverage state-of-the-art statistical analysis and visualization methods. S-PLUS 6 includes new capabilities for linear and nonlinear mixed-effects models, a major upgrade to survival analysis tools, two new libraries: missing data and robust statistics, and more.
Now, for the first time on UNIX you can access the powerful statistical and graphing techniques of S-PLUS through an intuitive, point-and-click graphical user interface. Easily import and export your data, run statistical analyses, and create insightful graphs. You can also integrate S-PLUS with your applications using CONNECT/Java. Let S-PLUS provide you with powerful statistical data mining and predictive modeling capabilities. Please click to http://www.insightful.com/
The apache web server is preinstalled with Solaris 9. To enable access, create an html page in a directory called public_html within your home directory. Please make sure that the public_html folder must in 755 mode .You can then access this web page via the following URL http://INSPEM1.upm.edu.my/~username/index.html Please make sure you use .html and not .htm as the file extension.
sazrol@INSPEM1%mkdir public_html
sazrol@INSPEM1%chmod 755 public_html
sazrol@INSPEM1%cd public_html
sazrol@INSPEM1%touch index.html