PAR Technologies V5 Using the ParaStation5 queuing facility, Using non-ParaStationapplications

Page 24

Using the ParaStation5 queuing facility

a predefined node list. If not defined, all currently known nodes are taken into account. Also, the variables PSI_NODES_SORT, PSI_LOOP_NODES_FIRST, PSI_EXCLUSIVE and PSI_OVERBOOK are observed. Based on these variables and the list of currently active processes, a sorted list of nodes is constructed, defining the final node list for this new task.

Beside this environment variables, node reservations for users and groups are also observed. See psiadmin(1).

In addition, only available nodes will be used to start up processes. Currently not available nodes will be ignored.

Obeying all these restrictions, the processes constructing a parallel task will be spawned on the nodes listed within the final node list. For SMP systems, all available CPUs (cores) on this node may be used for consecutive ranks, depending on the environment variable PSI_LOOP_NODES_FIRST.

For administrative tasks not requiring a dedicated CPU (core), e.g. processes spawned using pssh, other strategies take place. As this type of processes are intended to run on dedicated nodes, predefined by the user, the described procedure will be circumvented and the processes will be run on the user-defined nodes.

For a detailed discussion of placing processes within ParaStation5, please refer to process placement(7), ps_environment(5), pssh(8) and mpiexec(8).

5.4. Using the ParaStation5 queuing facility

ParaStation is able to queue task start requests if required resources are currently in use. This queuing facility is disabled by default. It can be enabled by each user independently. All requests of all users are held within one queue and managed on a first-come-first-serve strategy.

For details, refer to ParaStation5 User's Guide.

5.5. Exporting environment variables for a task

ParaStation by default exports only a limited set of environment variables to newly spawned processes, like HOME, USER, SHELL or TERM.

Additional variables can be exported using PSI_EXPORTS.

For a complete list of environment variables known to and exported automatically by ParaStation, refer to ps_environment(5) .

For more details, refer to ParaStation5 User's Guide.

5.6. Using non-ParaStationapplications

It is possible to run programs linked with 3rd party MPI libraries within the ParaStation environment. Currently supported MPI1 compatible libraries are:

MPIch using ch_p4 (mpirun_chp4)

MPIch using GM (mpirun_chgm)

InfiniPath (mpirun-ipath-ps)

MVAPIch (mpirun_openib)

QsNet MPI (mpirun_elan)

20

ParaStation5 Administrator's Guide

Image 24
Contents Administrators Guide ParaStation5 Administrators Guide Info@par-tec.comTable of Contents Problem different groups of nodes are seen as up or down Introduction History of ParaStationAbout this document Technical overview Runtime daemonLibraries Kernel modulesLicense Installation PrerequisitesHardware Directory structure SoftwareKernel version Installation via RPM packages Getting the ParaStation5 RPM packagesMan Mpi2, mpi2-intel, mpi2-pgi, mpi2-pscInstalling the RPMs Compiling the ParaStation5 packages from sourceFile Version Installing the documentation Etc/init.d/xinetd reloadParaStation entries Installing MPI Further steps# rpm -Uv psdoc-5.0.0-1.noarch.rpm # rpm -Uv psmpi2.5.0.0-1.i586.rpmUninstalling ParaStation5 ParaStation5 Administrators Guide Configuration Configuration of the ParaStation systemCopy template Define Number of nodesEnable optimized network drivers # /opt/parastation/bin/testconfigHostname id HWType runJob starter accounter Testing the installation # /opt/parastation/bin/psiadmin -s -c list # /opt/parastation/bin/testnodes -np nodesInsight ParaStation5 ParaStation5 pscom communication library# echo 10 /proc/sys/ps4/state/ResendTimeout Directory /proc/sys/ps4/state# cat /proc/sys/ps4/state/connections Controlling process placement Directory /proc/sys/ps4/localUsing the ParaStation5 queuing facility Using non-ParaStationapplicationsExporting environment variables for a task Controlling ParaStation5 communication paths Export LDPRELOAD=/opt/parastation/lib64/libp4tcp.soAuthentication within ParaStation5 Pspshm or PspsharedmemPSPP4S or PSPP4SOCK Export PSPLIB=/opt/parastation/lib64/libpscomopenib.soHomogeneous user ID space Single system viewParallel shell tool Nodes and CPUsIntegration with AFS Integrating external queuing systemsTok2env PSIRARGPRE0=/some/path/env2tokMulticasts Using ParaStation accounting Copying files in parallel# UseMCast Route add -net 224.0.0.0 netmask 240.0.0.0 dev ethXUsing ParaStation process pinning Using memory bindingChanging the default ports for psid8 Spawning processes belonging to all groupsPort Troubleshooting Problem psiadmin returns errorProblem node shown as down Problem cannot start parallel task Problem bad performanceProblem different groups of nodes are seen as up or down Problem cannot start process on frontendProblem pssh fails Problem psid does not startup, reports port in useProblem processes cannot access files on remote nodes Reference Pages ParaStation5 Administrators Guide InstallDir inst-dir , InstallationDir inst-dir Parastation.confDescription ParametersSetupscript StartscriptStopscript StatusscriptP4sock OpenibMvapi ElanAccounter NrOfNodes numNode node17 16 HWType ethernet p4sock starter yes runJobs no $GENERATE 1-96 node$0,2 $0SelectTime time DeadInterval numLogLevel num MCastGroup group-numCore size CPUTime timeDataSize size MemLock sizeProc Processes maxprocs CPUmap mapRdpTimeout ms StatusTimeout msRdpClosedTimeout ms RdpResendTimeout msErrors See alsoParaStation5 Administrators Guide Psiadmin SynopsisOptions Standard Error Standard InputStandard Output Extended descriptionExit AllAllproc cnt count Count hw hw DownHardware LoadRdp Summary max maxQuit Accounters nodes User nodesGroup nodes Maxproc nodesMaster nodes FreeOnSuspend nodesHandleOldBins nodes NodesSort nodesCpumap nodes Rlrss nodesRdpTimeout nodes StatusTimeout nodesRdpClosedTimeout nodes RdpResendTimeout nodesResolve nodes Restart nodesSelecttime time nodes Psiddebug mask nodesPattern Name Description HandleOldBins 0 1 nodes Rdpmaxretrans val nodes RdpTimeout ms nodes StatusTimeout ms nodesRdpClosedTimeout ms nodes RdpResendTimeout ms nodesFiles QuietNormal VerbosePsid Configfile=file Debug=levelLogfile=file Testconfig FilenameNum ? , --usage Show a help messageParaStation5 Administrators Guide Testnodes Np numCnt num MapParaStation5 Administrators Guide Testpse Testpse -npnumParaStation5 Administrators Guide P4stat SockNet ?,--helpParaStation5 Administrators Guide P4tcp AddDelete ParaStation5 Administrators Guide Psaccounter Pattern DescriptionDumpcore Coredir=dir?, --help Var/account/yyyymmdd Accounting files, one per dayPsaccview Lj,--ljobs Lu,--ltotuserLg,--ltotgroup Ls,--ltotsumCpuweight AqtimeCputime EndInitialization file Mlisten ParaStation5 Administrators Guide Appendix A. Quick Installation Guide # /opt/parastation/bin/psiadmin psiadmin add # chkconfig -a /etc/init.d/parastationTesting Appendix B. ParaStation license Page Page Page Building and installing ParaStation5 packages # psiadmin -sAppendix C. Upgrading ParaStation4 to ParaStation5 Changes to the runtime environmentPage Glossary ARPSee ParaStation Logger To share a common address space within a node ParaStation5 Administrators Guide