logo资料库

PBS Pro 9.3用户手册.pdf

第1页 / 共302页
第2页 / 共302页
第3页 / 共302页
第4页 / 共302页
第5页 / 共302页
第6页 / 共302页
第7页 / 共302页
第8页 / 共302页
资料共302页,剩余部分请下载后查看
Acknowledgements
Preface
Introduction
1.1 Book Organization
1.2 Supported Platforms
1.3 What is PBS Professional?
1.4 History of PBS
1.5 About the PBS Team
1.6 About Altair Engineering
1.7 Why Use PBS?
Concepts and Terms
2.1 PBS Components
2.2 Defining PBS Concepts and Terms
Getting Started With PBS
3.1 New Features in This Release
3.1.1 Job-Specific Staging and Execution Directories
3.1.2 Standing Reservations
3.2 New Features in Recent Releases
3.3 Deprecations
3.4 Using PBS
3.5 PBS Interfaces
3.6 User’s PBS Environment
3.7 Usernames Under PBS
3.8 Setting Up Your UNIX/Linux Environment
3.8.1 Setting MANPATH on SGI Systems
3.9 Setting Up Your Windows Environment
3.9.1 Windows User's HOMEDIR
3.9.2 Windows Usernames and Job Submission
3.9.3 Windows rhosts File
3.9.4 Windows Mapped Drives and PBS
3.10 Environment Variables
3.11 Temporary Scratch Space: TMPDIR
Submitting a PBS Job
4.1 Vnodes: Virtual Nodes
4.1.1 Relationship Between Hosts, Nodes, and Vnodes
4.1.2 Vnode Types
4.2 PBS Resources
4.2.0.1 Unset Resources
4.2.0.2 Resource Names and Values
4.2.1 Resource Types
4.2.2 Built-in Resources
4.3 PBS Jobs
4.3.1 Rules for Submitting Jobs
4.3.2 PBS Job Script
4.3.2.1 Specifying the Shell
4.3.2.2 PBS Directives
4.3.2.3 The User’s Tasks
4.3.3 Setting Job Attributes
4.4 Submitting a PBS Job
4.4.1 Submitting a Job Script
4.4.1.1 Overriding Directives
4.4.1.2 Submitting a Simple Job
4.4.1.3 Jobs Without a Job Script
4.4.1.4 Passing Arguments to Job Scripts
4.5 Requesting Resources
4.5.1 Allocation
4.5.2 Requesting Resources in Chunks
4.5.3 Requesting Job-wide Resources
4.5.4 Boolean Resources
4.5.5 Default Resources
4.5.6 Requesting Application Licenses
4.5.6.1 Floating Licenses
4.5.6.2 Node-locked Licenses
4.5.7 Requesting Scratch Space
4.5.8 Note About Submitting Jobs
4.5.9 Submitting Jobs with Resource Specification (Old Syntax)
4.5.10 Moving Jobs From One Queue to Another
4.5.11 Resource Request Conversion Dependent on Where Resources are Defined
4.5.12 Jobs Submitted with Undefined Resources
4.5.13 Limits on Resource Usage
4.5.14 Resource Permissions
4.6 Placing Jobs on Vnodes
4.6.1 Vnodes Allocated to a Job
4.6.2 PBS_NODEFILE
4.6.3 Resources Allocated from a Vnode
4.6.3.1 Resources Assigned to a Job
4.7 Submitting Jobs Using Select & Place: Examples
4.7.1 Examples Using Old Syntax
4.8 Backward Compatibility
4.8.1 Node Specification Conversion
4.8.2 Resource Specification Conversion
4.9 How PBS Parses a Job Script
4.10 A Sample PBS Job
4.11 Changing the Job’s PBS Directive
4.12 Windows Jobs
4.12.1 Submitting Windows Jobs
4.12.2 Passwords
4.12.2.1 Single-Signon Password Method
4.12.2.2 Per-job Password Method
4.13 Job Submission Options
4.13.1 Specifying Queue and/or Server
4.13.2 Redirecting Output and Error Files
4.13.3 Exporting Environment Variables
4.13.4 Expanding Environment Variables
4.13.5 Specifying Email Notification
4.13.6 Setting Email Recipient List
4.13.7 Specifying a Job Name
4.13.8 Marking a Job as “Rerunnable” or Not
4.13.9 Specifying Scripting Language to Use
4.13.10 Setting a Job’s Priority
4.13.11 Deferring Execution
4.13.12 Holding a Job (Delaying Execution)
4.13.13 Specifying Job Checkpoint Interval
4.13.13.1 Checkpointable Jobs
4.13.13.2 Checkpoint Interval
4.13.14 Specifying Job User ID
4.13.14.1 qsub -u: User ID with UNIX
4.13.14.2 qsub -u: User ID with Windows
4.13.15 Specifying Job Group ID
4.13.16 Specifying a Local Account
4.13.17 Merging Output and Error Files
4.13.18 Retaining Output and Error Files on Execution Host
4.13.19 Suppressing Job Identifier
4.13.20 Interactive-batch Jobs
4.14 Job Attributes
Using the xpbs GUI
5.1 Starting xpbs
5.1.1 Running xpbs Under UNIX
5.2 Using xpbs: Definitions of Terms
5.3 Introducing the xpbs Main Display
5.3.1 xpbs Menu Bar
5.3.2 xpbs Hosts Panel
5.3.3 xpbs Queues Panel
5.3.4 xpbs Jobs Panel
5.3.5 xpbs Info Panel
5.3.6 xpbs Keyboard Tips
5.4 Setting xpbs Preferences
5.5 Relationship Between PBS and xpbs
5.6 How to Submit a Job Using xpbs
5.7 Exiting xpbs
5.8 The xpbs Configuration File
5.9 xpbs Preferences
Checking Job / System Status
6.1 The qstat Command
6.1.1 Checking Job Status
6.1.2 Viewing Specific Information
6.1.3 Checking Server Status
6.1.4 Checking Queue Status
6.1.5 Viewing Job Information
6.1.6 List User-Specific Jobs
6.1.7 List Running Jobs
6.1.8 List Non-Running Jobs
6.1.9 Display Size in Gigabytes
6.1.10 Display Size in Megawords
6.1.11 List Hosts Assigned to Jobs
6.1.12 Display Job Comments
6.1.13 Display Queue Limits
6.1.14 Show State of Job, Job Array or Subjob
6.1.15 Viewing Job Status in Wide Format
6.1.16 Show state of Job Arrays
6.1.17 Print Job Array Percentage Completed
6.1.18 Getting Information on Jobs Moved to Another Server
6.1.19 Viewing Resources Allocated to a Job
6.2 Viewing Job / System Status with xpbs
6.3 The qselect Command
6.4 Selecting Jobs Using xpbs
6.5 Using xpbs TrackJob Feature
Working With PBS Jobs
7.1 Modifying Job Attributes
7.1.1 Changing the Selection Directive
7.1.2 Changing the Job-wide Limit
7.2 Holding and Releasing Jobs
7.3 Deleting Jobs
7.4 Sending Messages to Jobs
7.5 Sending Signals to Jobs
7.6 Changing Order of Jobs
7.7 Moving Jobs Between Queues
7.8 Converting a Job into a Reservation Job
Advanced PBS Features
8.1 New Features
8.1.1 Job-Specific Staging and Execution Directories
8.1.2 Standing Reservations
8.2 UNIX Job Exit Status
8.3 Changing UNIX Job umask
8.4 Requesting qsub Wait for Job Completion
8.5 Specifying Job Dependencies
8.5.1 Job Array Dependencies
8.6 Delivery of Output Files
8.7 Input/Output File Staging
8.7.1 Staging and Execution Directory: User’s Home vs. Job-specific
8.7.2 Using Job-specific Staging and Execution Directories
8.7.2.1 Setting the Job’s Staging and Execution Directory
8.7.2.2 The Job’s jobdir Attribute and the PBS_JOBDIR Environ ment Variable
8.7.3 Attributes and Environment Variables Affecting Staging
8.7.4 Specifying Files To Be Staged In or Staged Out
8.7.4.1 Copying Directories Into and Out Of the Staging and Execu tion Directory
8.7.4.2 Wildcards In File Staging
8.7.4.3 Caveats
8.7.4.4 Output Filenames
8.7.5 Example of Using Job-specific Staging and Execution Directories
8.7.6 Summary of the Job’s Lifecycle
8.7.7 Detailed Description of Job’s Lifecycle
8.7.7.1 Creation of TMPDIR
8.7.7.2 Choice of Staging and Execution Directories
8.7.7.3 Setting Environment Variables and Attributes
8.7.7.4 Staging Files Into Staging and Execution Directories
8.7.7.5 Running the Prologue
8.7.7.6 Job Execution
8.7.7.7 Standard Out and Standard Error
8.7.7.8 Running the Epilogue
8.7.7.9 Staging Files Out and Removing Execution Directory
8.7.7.10 Removing TMPDIRs
8.7.8 Staging with Job Arrays
8.7.9 Using xpbs for File Staging
8.7.10 Stagein and Stageout Failure
8.8 The pbsdsh Command
8.9 Advance and Standing Reservation of Resources
8.9.1 Introduction to Creating and Using Reservations
8.9.2 Creating Advance Reservations
8.9.2.1 Examples of Creating Advance Reservations
8.9.3 Creating Standing Reservations
8.9.3.1 Setting Reservation Start Time and Duration
8.9.3.2 Requirements for Creating Standing Reservations
8.9.3.3 Examples of Creating Standing Reservations
8.9.4 The pbs_rsub Command
8.9.4.1 Getting Confirmation of a Reservation
8.9.5 Viewing the Status of a Reservation
8.9.5.1 Examples of Viewing Reservation Status Using pbs_rstat
8.9.6 Deleting Reservations
8.9.7 Submitting a Job to a Reservation
8.9.7.1 Running Jobs in a Reservation
8.9.7.2 Access to Reservations
8.9.7.3 Viewing Status of a Job Submitted to a Reservation
8.9.8 Reservation Caveats and Errors
8.9.8.1 Setting the Submission Host’s Time Zone
8.9.8.2 Reservation Errors
8.9.8.3 Time Required Between Reservations
8.9.9 Reservation Information in the Accounting Log
8.10 Dedicated Time
8.11 Using Comprehensive System Accounting
8.12 Running PBS in a UNIX DCE Environment
8.13 Running PBS in a UNIX Kerberos Environment
8.14 Support for Large Page Mode on AIX
Job Arrays
9.1 Definitions
9.1.1 Description
9.1.2 Identifier Syntax
9.2 qsub: Submitting a Job Array
9.2.1 Interactive Job Submission
9.3 Job Array Attributes
9.4 Job Array States
9.4.1 Subjob States
9.5 PBS Environmental Variables
9.6 File Staging
9.6.1 Specifying Files To Be Staged In or Staged Out
9.6.1.1 Scripts
9.6.1.2 Output Filenames
9.6.2 Job Array Staging Syntax on Windows
9.7 PBS Commands
9.7.1 PBS Commands Taking Job Arrays as Arguments
9.7.2 qstat: Status of a Job Array
9.7.3 qdel: Deleting a Job Array
9.7.4 qalter: Altering a Job Array
9.7.5 qorder: Ordering Job Arrays in the Queue
9.7.6 qmove: Moving a Job Array
9.7.7 qhold: Holding a Job Array
9.7.8 qrls: Releasing a Job Array
9.7.9 qrerun: Requeueing a Job Array
9.7.10 qrun: Running a Job Array
9.7.11 tracejob on Job Arrays
9.7.12 qsig: Signaling a Job Array
9.7.13 qmsg: Sending Messages
9.8 Other PBS Commands Supported for Job Arrays
9.8.1 qselect: Selection of Job Arrays
9.9 Job Arrays and xpbs
9.10 More on Job Arrays
9.10.1 Job Array Run Limits
9.10.2 Starving
9.10.3 Job Array Dependencies
9.10.4 Accounting
9.10.5 Checkpointing
9.10.6 Prologues and Epilogues
9.10.7 Job Array Exit Status
9.10.8 Scheduling Job Arrays
9.10.8.1 Preemption
9.10.8.2 Peer Scheduling
9.10.8.3 Fairshare
9.10.8.4 Placement Sets and Node Grouping
Multiprocessor Jobs
10.1 Job Placement
10.2 Submitting SMP Jobs
10.3 Submitting MPI Jobs
10.3.1 The mpiprocs Resource
10.4 OpenMP Jobs with PBS
10.5 Hybrid MPI-OpenMP Jobs
10.6 MPI Jobs with PBS
10.6.1 MPICH Jobs With PBS
10.6.2 MPI Jobs Using LAM MPI
10.6.3 MPI Jobs Using AIX, POE
10.6.3.1 Examples Using poe
10.6.3.2 If Your Complex Contains Machines Not on the HPS
10.6.4 PBS MPI Jobs on HP-UX and Linux
10.6.5 PBS Jobs with MPICH-GM's mpirun Using rsh/ssh (mpirun.ch_gm)
10.6.5.1 Options
10.6.5.2 Examples
10.6.6 PBS Jobs with MPICH-MX's mpirun Using rsh/ssh (mpirun.ch_mx)
10.6.6.1 Options
10.6.6.2 Examples
10.6.7 PBS Jobs with MPICH-GM's mpirun Using MPD (mpirun.mpd)
10.6.7.1 Options
10.6.7.2 MPD Startup and Shutdown
10.6.7.3 Examples
10.6.8 PBS Jobs with MPICH-MX's mpirun Using MPD (mpirun.mpd)
10.6.8.1 Options
10.6.8.2 MPD Startup and Shutdown
10.6.8.3 Examples
10.6.9 PBS Jobs with MPICH2's mpirun
10.6.9.1 Options
10.6.9.2 MPD Startup and Shutdown
10.6.9.3 Examples
10.6.10 PBS Jobs with Intel MPI's mpirun
10.6.10.1 Options
10.6.10.2 MPD Startup and Shutdown
10.6.10.3 Examples
10.6.11 PBS Jobs with MVAPICH1's mpirun
10.6.11.1 Options
10.6.11.2 Examples
10.6.12 PBS Jobs with MVAPICH2's mpiexec
10.6.12.1 Options
10.6.12.2 MPD Startup and Shutdown
10.6.12.3 Examples
10.6.13 PBS Jobs with HP MPI
10.7 MPI Jobs on the Altix
10.7.1 Jobs on an Altix Running ProPack 4/5
10.8 PVM Jobs with PBS
10.9 Checkpointing SGI MPI Jobs
10.9.1 Jobs on an Altix
Appendix A: PBS Environment Variables
Appendix B: Converting From NQS to PBS
Appendix C: License Agreement
Index
PBS Professional 9.2 USER’S GUIDE A division of
Altair® PBS Professional™ 9.2 User’s Guide UNIX®, Linux® and Windows®
TM User’s Guide PBS Professional Altair PBS Professional ® Edited by: Anne Urban Copyright © 2004-2008 Altair Engineering, Inc. All rights reserved. 9.2, Updated: 5/19/08 TM Trademark Acknowledgements : “PBS Professional”, “PBS Pro”, “Porta- ble Batch System” and the PBS Juggler logo are trademarks of Altair Engi- neering, Inc. All other trademarks are the property of their respective owners. For more information, copies of these books, and for product sales, contact Altair at: Web: www.altair.com www.pbspro.com Email: sales@pbspro.com Technical Support Table 1: Location Telephone e-mail North America China France Germany India Italy Japan Korea Scandinavia UK +1 248 614 2425 +86 (0)21 5393 0011 +33 (0)1 4133 0990 +49 (0)7031 6208 22 +91 80 658 8540 +91 80 658 8542 +39 0832 315573 +39 800 905595 +81 3 5396 1341 +82 31 728 8600 +46 (0)46 286 2050 +44 (0) 2476 323 600 pbssupport@altair.com support@altair.com.cn francesupport@altair.com hwsupport@altair.de pbs-sup- port@india.altair.com support@altairtorino.it pbs@altairjp.co.jp support@altair.co.kr support@altair.se support@uk.altair.com This document is proprietary information of Altair Engineering, Inc.
Table of Contents Acknowledgements Preface 1 vii ix Introduction 1 1.1 Book Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Supported Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 What is PBS Professional? . . . . . . . . . . . . . . . . . . . . . 3 1.4 History of PBS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.5 About the PBS Team . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 About Altair Engineering . . . . . . . . . . . . . . . . . . . . . . 5 1.7 Why Use PBS? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 9 2.1 PBS Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Defining PBS Concepts and Terms . . . . . . . . . . . . . 12 19 3.1 New Features in This Release . . . . . . . . . . . . . . . . . 19 2 Concepts and Terms 3 Getting Started With PBS PBS Professional 9.2 User’s Guide iii
Table of Contents 4 Submitting a PBS Job 3.2 New Features in Recent Releases . . . . . . . . . . . . . . 20 3.3 Deprecations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.4 Using PBS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.5 PBS Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.6 User’s PBS Environment . . . . . . . . . . . . . . . . . . . . . 23 3.7 Usernames Under PBS. . . . . . . . . . . . . . . . . . . . . . . 24 3.8 Setting Up Your UNIX/Linux Environment . . . . . . 24 3.9 Setting Up Your Windows Environment. . . . . . . . . 26 3.10 Environment Variables . . . . . . . . . . . . . . . . . . . . . . 29 3.11 Temporary Scratch Space: TMPDIR . . . . . . . . . . . . 30 33 4.1 Vnodes: Virtual Nodes. . . . . . . . . . . . . . . . . . . . . . . 33 4.2 PBS Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 PBS Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.3 4.4 Submitting a PBS Job . . . . . . . . . . . . . . . . . . . . . . . 44 4.5 Requesting Resources . . . . . . . . . . . . . . . . . . . . . . . 46 Placing Jobs on Vnodes . . . . . . . . . . . . . . . . . . . . . . 56 4.6 4.7 Submitting Jobs Using Select & Place: Examples. . 60 4.8 Backward Compatibility . . . . . . . . . . . . . . . . . . . . . 65 4.9 How PBS Parses a Job Script. . . . . . . . . . . . . . . . . . 68 4.10 A Sample PBS Job . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.11 Changing the Job’s PBS Directive. . . . . . . . . . . . . . 70 4.12 Windows Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.13 Job Submission Options. . . . . . . . . . . . . . . . . . . . . . 74 4.14 Job Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 97 5.1 Starting xpbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.2 Using xpbs: Definitions of Terms . . . . . . . . . . . . . . 99 5.3 Introducing the xpbs Main Display . . . . . . . . . . . . 100 5.4 Setting xpbs Preferences . . . . . . . . . . . . . . . . . . . . 106 5.5 Relationship Between PBS and xpbs. . . . . . . . . . . 106 5.6 How to Submit a Job Using xpbs. . . . . . . . . . . . . . 108 5 Using the xpbs GUI iv PBS Professional 9.2 User’s Guide
Table of Contents 6 Checking Job / System Status 7 Working With PBS Jobs 5.7 5.8 5.9 Exiting xpbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 The xpbs Configuration File. . . . . . . . . . . . . . . . . . 111 xpbs Preferences. . . . . . . . . . . . . . . . . . . . . . . . . . . 111 115 6.1 The qstat Command . . . . . . . . . . . . . . . . . . . . . . . . 115 6.2 Viewing Job / System Status with xpbs . . . . . . . . . 128 6.3 The qselect Command . . . . . . . . . . . . . . . . . . . . . . 129 6.4 Selecting Jobs Using xpbs . . . . . . . . . . . . . . . . . . . 134 6.5 Using xpbs TrackJob Feature . . . . . . . . . . . . . . . . . 135 137 7.1 Modifying Job Attributes . . . . . . . . . . . . . . . . . . . . 137 7.2 Holding and Releasing Jobs . . . . . . . . . . . . . . . . . . 141 7.3 Deleting Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 7.4 Sending Messages to Jobs . . . . . . . . . . . . . . . . . . . 144 7.5 Sending Signals to Jobs . . . . . . . . . . . . . . . . . . . . . 145 7.6 Changing Order of Jobs . . . . . . . . . . . . . . . . . . . . . 146 7.7 Moving Jobs Between Queues . . . . . . . . . . . . . . . . 148 7.8 Converting a Job into a Reservation Job . . . . . . . . 149 151 8.1 New Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 8.2 UNIX Job Exit Status . . . . . . . . . . . . . . . . . . . . . . . 152 8.3 Changing UNIX Job umask . . . . . . . . . . . . . . . . . . 152 8.4 Requesting qsub Wait for Job Completion. . . . . . . 153 8.5 Specifying Job Dependencies. . . . . . . . . . . . . . . . . 153 8.6 Delivery of Output Files. . . . . . . . . . . . . . . . . . . . . 157 Input/Output File Staging . . . . . . . . . . . . . . . . . . . . 158 8.7 8.8 The pbsdsh Command . . . . . . . . . . . . . . . . . . . . . . 171 8.9 Advance and Standing Reservation of Resources . 172 8.10 Dedicated Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 8.11 Using Comprehensive System Accounting . . . . . . 197 8.12 Running PBS in a UNIX DCE Environment . . . . . 198 8 Advanced PBS Features PBS Professional 9.2 User’s Guide v
Table of Contents 9 10 Multiprocessor Jobs 8.13 Running PBS in a UNIX Kerberos Environment . 199 8.14 Support for Large Page Mode on AIX. . . . . . . . . . 199 Job Arrays 201 9.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 qsub: Submitting a Job Array. . . . . . . . . . . . . . . . . 203 9.2 Job Array Attributes. . . . . . . . . . . . . . . . . . . . . . . . 205 9.3 9.4 Job Array States . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 PBS Environmental Variables . . . . . . . . . . . . . . . . 207 9.5 9.6 File Staging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 9.7 PBS Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 9.8 Other PBS Commands Supported for Job Arrays . 219 9.9 Job Arrays and xpbs. . . . . . . . . . . . . . . . . . . . . . . . 220 9.10 More on Job Arrays . . . . . . . . . . . . . . . . . . . . . . . . 220 223 10.1 Job Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 10.2 Submitting SMP Jobs. . . . . . . . . . . . . . . . . . . . . . . 224 10.3 Submitting MPI Jobs . . . . . . . . . . . . . . . . . . . . . . . 224 10.4 OpenMP Jobs with PBS. . . . . . . . . . . . . . . . . . . . . 226 10.5 Hybrid MPI-OpenMP Jobs . . . . . . . . . . . . . . . . . . 226 10.6 MPI Jobs with PBS . . . . . . . . . . . . . . . . . . . . . . . . 228 10.7 MPI Jobs on the Altix . . . . . . . . . . . . . . . . . . . . . . 258 10.8 PVM Jobs with PBS. . . . . . . . . . . . . . . . . . . . . . . . 260 10.9 Checkpointing SGI MPI Jobs . . . . . . . . . . . . . . . . 261 11 Appendix A: PBS Environment Variables 263 12 Appendix B: Converting From NQS to PBS 267 13 Appendix C: License Agreement Index 269 279 vi PBS Professional 9.2 User’s Guide
分享到:
收藏