Acknowledgements
Preface
Introduction
1.1 Book Organization
1.2 Supported Platforms
1.3 What is PBS Professional?
1.4 History of PBS
1.5 About the PBS Team
1.6 About Altair Engineering
1.7 Why Use PBS?
Concepts and Terms
2.1 PBS Components
2.2 Defining PBS Concepts and Terms
Getting Started With PBS
3.1 New Features in This Release
3.1.1 Job-Specific Staging and Execution Directories
3.1.2 Standing Reservations
3.2 New Features in Recent Releases
3.3 Deprecations
3.4 Using PBS
3.5 PBS Interfaces
3.6 User’s PBS Environment
3.7 Usernames Under PBS
3.8 Setting Up Your UNIX/Linux Environment
3.8.1 Setting MANPATH on SGI Systems
3.9 Setting Up Your Windows Environment
3.9.1 Windows User's HOMEDIR
3.9.2 Windows Usernames and Job Submission
3.9.3 Windows rhosts File
3.9.4 Windows Mapped Drives and PBS
3.10 Environment Variables
3.11 Temporary Scratch Space: TMPDIR
Submitting a PBS Job
4.1 Vnodes: Virtual Nodes
4.1.1 Relationship Between Hosts, Nodes, and Vnodes
4.1.2 Vnode Types
4.2 PBS Resources
4.2.0.1 Unset Resources
4.2.0.2 Resource Names and Values
4.2.1 Resource Types
4.2.2 Built-in Resources
4.3 PBS Jobs
4.3.1 Rules for Submitting Jobs
4.3.2 PBS Job Script
4.3.2.1 Specifying the Shell
4.3.2.2 PBS Directives
4.3.2.3 The User’s Tasks
4.3.3 Setting Job Attributes
4.4 Submitting a PBS Job
4.4.1 Submitting a Job Script
4.4.1.1 Overriding Directives
4.4.1.2 Submitting a Simple Job
4.4.1.3 Jobs Without a Job Script
4.4.1.4 Passing Arguments to Job Scripts
4.5 Requesting Resources
4.5.1 Allocation
4.5.2 Requesting Resources in Chunks
4.5.3 Requesting Job-wide Resources
4.5.4 Boolean Resources
4.5.5 Default Resources
4.5.6 Requesting Application Licenses
4.5.6.1 Floating Licenses
4.5.6.2 Node-locked Licenses
4.5.7 Requesting Scratch Space
4.5.8 Note About Submitting Jobs
4.5.9 Submitting Jobs with Resource Specification (Old Syntax)
4.5.10 Moving Jobs From One Queue to Another
4.5.11 Resource Request Conversion Dependent on Where Resources are Defined
4.5.12 Jobs Submitted with Undefined Resources
4.5.13 Limits on Resource Usage
4.5.14 Resource Permissions
4.6 Placing Jobs on Vnodes
4.6.1 Vnodes Allocated to a Job
4.6.2 PBS_NODEFILE
4.6.3 Resources Allocated from a Vnode
4.6.3.1 Resources Assigned to a Job
4.7 Submitting Jobs Using Select & Place: Examples
4.7.1 Examples Using Old Syntax
4.8 Backward Compatibility
4.8.1 Node Specification Conversion
4.8.2 Resource Specification Conversion
4.9 How PBS Parses a Job Script
4.10 A Sample PBS Job
4.11 Changing the Job’s PBS Directive
4.12 Windows Jobs
4.12.1 Submitting Windows Jobs
4.12.2 Passwords
4.12.2.1 Single-Signon Password Method
4.12.2.2 Per-job Password Method
4.13 Job Submission Options
4.13.1 Specifying Queue and/or Server
4.13.2 Redirecting Output and Error Files
4.13.3 Exporting Environment Variables
4.13.4 Expanding Environment Variables
4.13.5 Specifying Email Notification
4.13.6 Setting Email Recipient List
4.13.7 Specifying a Job Name
4.13.8 Marking a Job as “Rerunnable” or Not
4.13.9 Specifying Scripting Language to Use
4.13.10 Setting a Job’s Priority
4.13.11 Deferring Execution
4.13.12 Holding a Job (Delaying Execution)
4.13.13 Specifying Job Checkpoint Interval
4.13.13.1 Checkpointable Jobs
4.13.13.2 Checkpoint Interval
4.13.14 Specifying Job User ID
4.13.14.1 qsub -u: User ID with UNIX
4.13.14.2 qsub -u: User ID with Windows
4.13.15 Specifying Job Group ID
4.13.16 Specifying a Local Account
4.13.17 Merging Output and Error Files
4.13.18 Retaining Output and Error Files on Execution Host
4.13.19 Suppressing Job Identifier
4.13.20 Interactive-batch Jobs
4.14 Job Attributes
Using the xpbs GUI
5.1 Starting xpbs
5.1.1 Running xpbs Under UNIX
5.2 Using xpbs: Definitions of Terms
5.3 Introducing the xpbs Main Display
5.3.1 xpbs Menu Bar
5.3.2 xpbs Hosts Panel
5.3.3 xpbs Queues Panel
5.3.4 xpbs Jobs Panel
5.3.5 xpbs Info Panel
5.3.6 xpbs Keyboard Tips
5.4 Setting xpbs Preferences
5.5 Relationship Between PBS and xpbs
5.6 How to Submit a Job Using xpbs
5.7 Exiting xpbs
5.8 The xpbs Configuration File
5.9 xpbs Preferences
Checking Job / System Status
6.1 The qstat Command
6.1.1 Checking Job Status
6.1.2 Viewing Specific Information
6.1.3 Checking Server Status
6.1.4 Checking Queue Status
6.1.5 Viewing Job Information
6.1.6 List User-Specific Jobs
6.1.7 List Running Jobs
6.1.8 List Non-Running Jobs
6.1.9 Display Size in Gigabytes
6.1.10 Display Size in Megawords
6.1.11 List Hosts Assigned to Jobs
6.1.12 Display Job Comments
6.1.13 Display Queue Limits
6.1.14 Show State of Job, Job Array or Subjob
6.1.15 Viewing Job Status in Wide Format
6.1.16 Show state of Job Arrays
6.1.17 Print Job Array Percentage Completed
6.1.18 Getting Information on Jobs Moved to Another Server
6.1.19 Viewing Resources Allocated to a Job
6.2 Viewing Job / System Status with xpbs
6.3 The qselect Command
6.4 Selecting Jobs Using xpbs
6.5 Using xpbs TrackJob Feature
Working With PBS Jobs
7.1 Modifying Job Attributes
7.1.1 Changing the Selection Directive
7.1.2 Changing the Job-wide Limit
7.2 Holding and Releasing Jobs
7.3 Deleting Jobs
7.4 Sending Messages to Jobs
7.5 Sending Signals to Jobs
7.6 Changing Order of Jobs
7.7 Moving Jobs Between Queues
7.8 Converting a Job into a Reservation Job
Advanced PBS Features
8.1 New Features
8.1.1 Job-Specific Staging and Execution Directories
8.1.2 Standing Reservations
8.2 UNIX Job Exit Status
8.3 Changing UNIX Job umask
8.4 Requesting qsub Wait for Job Completion
8.5 Specifying Job Dependencies
8.5.1 Job Array Dependencies
8.6 Delivery of Output Files
8.7 Input/Output File Staging
8.7.1 Staging and Execution Directory: User’s Home vs. Job-specific
8.7.2 Using Job-specific Staging and Execution Directories
8.7.2.1 Setting the Job’s Staging and Execution Directory
8.7.2.2 The Job’s jobdir Attribute and the PBS_JOBDIR Environ ment Variable
8.7.3 Attributes and Environment Variables Affecting Staging
8.7.4 Specifying Files To Be Staged In or Staged Out
8.7.4.1 Copying Directories Into and Out Of the Staging and Execu tion Directory
8.7.4.2 Wildcards In File Staging
8.7.4.3 Caveats
8.7.4.4 Output Filenames
8.7.5 Example of Using Job-specific Staging and Execution Directories
8.7.6 Summary of the Job’s Lifecycle
8.7.7 Detailed Description of Job’s Lifecycle
8.7.7.1 Creation of TMPDIR
8.7.7.2 Choice of Staging and Execution Directories
8.7.7.3 Setting Environment Variables and Attributes
8.7.7.4 Staging Files Into Staging and Execution Directories
8.7.7.5 Running the Prologue
8.7.7.6 Job Execution
8.7.7.7 Standard Out and Standard Error
8.7.7.8 Running the Epilogue
8.7.7.9 Staging Files Out and Removing Execution Directory
8.7.7.10 Removing TMPDIRs
8.7.8 Staging with Job Arrays
8.7.9 Using xpbs for File Staging
8.7.10 Stagein and Stageout Failure
8.8 The pbsdsh Command
8.9 Advance and Standing Reservation of Resources
8.9.1 Introduction to Creating and Using Reservations
8.9.2 Creating Advance Reservations
8.9.2.1 Examples of Creating Advance Reservations
8.9.3 Creating Standing Reservations
8.9.3.1 Setting Reservation Start Time and Duration
8.9.3.2 Requirements for Creating Standing Reservations
8.9.3.3 Examples of Creating Standing Reservations
8.9.4 The pbs_rsub Command
8.9.4.1 Getting Confirmation of a Reservation
8.9.5 Viewing the Status of a Reservation
8.9.5.1 Examples of Viewing Reservation Status Using pbs_rstat
8.9.6 Deleting Reservations
8.9.7 Submitting a Job to a Reservation
8.9.7.1 Running Jobs in a Reservation
8.9.7.2 Access to Reservations
8.9.7.3 Viewing Status of a Job Submitted to a Reservation
8.9.8 Reservation Caveats and Errors
8.9.8.1 Setting the Submission Host’s Time Zone
8.9.8.2 Reservation Errors
8.9.8.3 Time Required Between Reservations
8.9.9 Reservation Information in the Accounting Log
8.10 Dedicated Time
8.11 Using Comprehensive System Accounting
8.12 Running PBS in a UNIX DCE Environment
8.13 Running PBS in a UNIX Kerberos Environment
8.14 Support for Large Page Mode on AIX
Job Arrays
9.1 Definitions
9.1.1 Description
9.1.2 Identifier Syntax
9.2 qsub: Submitting a Job Array
9.2.1 Interactive Job Submission
9.3 Job Array Attributes
9.4 Job Array States
9.4.1 Subjob States
9.5 PBS Environmental Variables
9.6 File Staging
9.6.1 Specifying Files To Be Staged In or Staged Out
9.6.1.1 Scripts
9.6.1.2 Output Filenames
9.6.2 Job Array Staging Syntax on Windows
9.7 PBS Commands
9.7.1 PBS Commands Taking Job Arrays as Arguments
9.7.2 qstat: Status of a Job Array
9.7.3 qdel: Deleting a Job Array
9.7.4 qalter: Altering a Job Array
9.7.5 qorder: Ordering Job Arrays in the Queue
9.7.6 qmove: Moving a Job Array
9.7.7 qhold: Holding a Job Array
9.7.8 qrls: Releasing a Job Array
9.7.9 qrerun: Requeueing a Job Array
9.7.10 qrun: Running a Job Array
9.7.11 tracejob on Job Arrays
9.7.12 qsig: Signaling a Job Array
9.7.13 qmsg: Sending Messages
9.8 Other PBS Commands Supported for Job Arrays
9.8.1 qselect: Selection of Job Arrays
9.9 Job Arrays and xpbs
9.10 More on Job Arrays
9.10.1 Job Array Run Limits
9.10.2 Starving
9.10.3 Job Array Dependencies
9.10.4 Accounting
9.10.5 Checkpointing
9.10.6 Prologues and Epilogues
9.10.7 Job Array Exit Status
9.10.8 Scheduling Job Arrays
9.10.8.1 Preemption
9.10.8.2 Peer Scheduling
9.10.8.3 Fairshare
9.10.8.4 Placement Sets and Node Grouping
Multiprocessor Jobs
10.1 Job Placement
10.2 Submitting SMP Jobs
10.3 Submitting MPI Jobs
10.3.1 The mpiprocs Resource
10.4 OpenMP Jobs with PBS
10.5 Hybrid MPI-OpenMP Jobs
10.6 MPI Jobs with PBS
10.6.1 MPICH Jobs With PBS
10.6.2 MPI Jobs Using LAM MPI
10.6.3 MPI Jobs Using AIX, POE
10.6.3.1 Examples Using poe
10.6.3.2 If Your Complex Contains Machines Not on the HPS
10.6.4 PBS MPI Jobs on HP-UX and Linux
10.6.5 PBS Jobs with MPICH-GM's mpirun Using rsh/ssh (mpirun.ch_gm)
10.6.5.1 Options
10.6.5.2 Examples
10.6.6 PBS Jobs with MPICH-MX's mpirun Using rsh/ssh (mpirun.ch_mx)
10.6.6.1 Options
10.6.6.2 Examples
10.6.7 PBS Jobs with MPICH-GM's mpirun Using MPD (mpirun.mpd)
10.6.7.1 Options
10.6.7.2 MPD Startup and Shutdown
10.6.7.3 Examples
10.6.8 PBS Jobs with MPICH-MX's mpirun Using MPD (mpirun.mpd)
10.6.8.1 Options
10.6.8.2 MPD Startup and Shutdown
10.6.8.3 Examples
10.6.9 PBS Jobs with MPICH2's mpirun
10.6.9.1 Options
10.6.9.2 MPD Startup and Shutdown
10.6.9.3 Examples
10.6.10 PBS Jobs with Intel MPI's mpirun
10.6.10.1 Options
10.6.10.2 MPD Startup and Shutdown
10.6.10.3 Examples
10.6.11 PBS Jobs with MVAPICH1's mpirun
10.6.11.1 Options
10.6.11.2 Examples
10.6.12 PBS Jobs with MVAPICH2's mpiexec
10.6.12.1 Options
10.6.12.2 MPD Startup and Shutdown
10.6.12.3 Examples
10.6.13 PBS Jobs with HP MPI
10.7 MPI Jobs on the Altix
10.7.1 Jobs on an Altix Running ProPack 4/5
10.8 PVM Jobs with PBS
10.9 Checkpointing SGI MPI Jobs
10.9.1 Jobs on an Altix
Appendix A: PBS Environment Variables
Appendix B: Converting From NQS to PBS
Appendix C: License Agreement
Index