qprep - Translate job-submission scripts from a platform independent format to the local, platform-dependent format.
qprep [key1=value1[ key2=value2 ...]] filename
qprep provides users with a uniform, site-independent command interface to the local queuing system, eliminating the need to learn the myriad syntactical variations that appear at different institutions to perform essentially identical job-submission tasks.
qprep translates a script from a site-independent format (described below) into a site-dependent format, typically one using NQS (Network Queuing System), PBS (Portable Batch System), or a similar queuing system. The translated script is written to a new file, and may optionally be submitted to a queue. Scripts written to the qprep standard will be usable at any site where qprep is implemented, regardless of the target queuing system.
Options to qprep are given in the form of key-value pairs, called ``directives''. These may be given either at the command line, seperated by white space, or in the preamble to a job script.
Job scripts contain a preamble, similar to those found in scripts processed by native queuing system. The preamble contains directives that control the resource limits and other attributes of the job. qprep directives in the preamble have this form:
#PSTQ key=value
The # character denotes the line is a script comment, the string PSTQ alerts qprep that the line is a directive that it needs to process, and the actual directive is in the form of a key=value pair, whose possible values are described below.
The preamble ends with this line:
#PSTQ END_OF_PREAMBLE
Qprep considers all lines following the first line matching the above string to follow the preamble. Qprep simply passes lines occuring after this directive to the output script without applying translation or interpretation.
By default qprep will also consider the first non-blank, non-comment line to end the preamble. See also the 'explicit_preamble' directive.
These directives specify the behavior of qprep upon translating a script. Some of these options only make sense if they occur on the qprep command-line. These options are not usually explicitly translated to the target queue system; they simply control behaviors of qprep.
Do not write the job identifier to stdout when the job is submitted to a queue.
Write the translated script to outfile. If this option is not specified, the translated script is written to filename.pst where filename is the script argument to qprep. For this option to work properly it must occur on the command line, not inside a script. If this directive occurs inside a script, it changes the contents of the parameter ``out_script'' but does not change the name of the actual output filename. That would effectively change which script gets submitted, if the submit=y directive occurs.
After the script has been translated, submit the new script to the queue.
Erase the translated script once it has been submitted to the queue. This action is only taken if used in conjunction with submit=y.
Include extra comments in the output script that could help the user understand how qprep translates input to the script format of the target queue system. The default value is ``n'', which tells qprep not to produce this auxiliary information. The value ``y'' (lower case) causes qprep to include, as comments, the original input lines. The value ``Y'' (upper case) causes qprep to include yet more information.
Set the verbosity of qprep. This is useful for debugging qprep itself, not something a typical user would find useful.
Flag indicating whether the END_OF_PREAMBLE directive is required. If explicit_preamble is 'n' then the first non-blank, non-comment line is considered to be the end-of-preamble.
NOTE: In the context of the preamble, qprep considers a comment line to be a line that begins with the PST qprep comment string, which is a '#', and it setable by the preamble_comment entry in the state table. Note that this could differ from the comment character for the script language.
Default value is 'n'.
These directives control the environment in which the script generated by qprep operates.
Charge the job to account account_string.
Run the job as the specified user. The userlist has the format:
user[@host][,user[@host], ...]
This allows the job to run as different users depending on the host that runs the job. Note, userlist is specified with no white space.
If export_vars is set to the string ``y'', environment variables are exported from the shell in which qprep is run to the script's environment.
Define the shell that will be used to run the script.
These directives control stdout, stderr, and mail reporting for the job.
Define the file to which stderr is written. This may be the same as the filename for stdout (see below).
Define the file to which stdout is written. This may be the same as the filename for stderr.
Keep either stdout or stderr, or both, on the host where the script was executed. See COMMAND NOTES for a comment on the comma notation used above.
Send mail at different occurrences. The user may specify multiple triggers separated by commas. See COMMAND NOTES for a discussion of the argument notation.
a: send mail when a job is aborted
b: send mail when a job begins execution
e: send mail when a job ends
r: send mail when a job is rerun
t: send mail when a job is routed
Specifies a comma-separated list (no white space) of users to whom mail is sent. Events that cause mail to be sent are set with mail=key.
These items specify particulars of how and where the job is run.
Specifies the queue to which the job is submitted.
Specifies the parallel environment to use to execute the job.
Specifies a name to associate with the job.
Defines the checkpointing characteristics of the job.
n: never checkpoint
s: checkpoint when the queue is shut down
number: checkpoint every (number) minutes.
NOTE: Not all platforms support checkpointing.
These items specify resource limits. All times are in the format hours:minutes, where hours and minutes are numbers. All sizes are in megabytes (MB).
Limits can be specified per-process (a single process within the script) or per-request (the total of all the processes within a script).
Specifies the number of nodes that are simultaneously required by the script. Default is nodes=1.
Specifies the number of cpus per node that are simultaneously required by the script. Default is cpus_per_node=1.
Specifies the maximum wall_time to be allotted for the script execution.
Specifies the maximum CPU time used by any single process within the script.
Specifies the total CPU time used by all single processes within the script.
Specifies the maximum total size, in megabytes, of files written by a single process within the script.
Specifies the maximum total size, in megabytes, of all files written by all processes within a script.
This set of keys (tapemta, tapemtb, ...) specifies the maximum number of tape drives required in the corresponding device class (a,b, ...). Tape drive device groups can be ascertained using the tpstat(1) command.
Specifies the nice level at which the script is run when resources are shared. See nice(1) for a description of nice levels.
Specifies the maximum memory used by a single process within the script. size is specified in megabytes (MB). If the process does not share memory with other processes, this option does not need to be specified.
Specifies the total memory used by all processes within the script. size is specified in megabytes (MB). If the request does not share memory with other requests, this option does not need to be specified.
Sometimes it is necessary for a job to depend in some way upon another job. qprep allows jobs to depend upon one another in three ways:
For after and before, dependency can be specified on jobs beginning, and ending (with error conditions taken into account). This allows robust scripting in which different jobs run depending on the successful or unsuccessful completion of another job.
The syntax here is very nearly that specified by PBS, which inspired these commands.
Synchronize (number) jobs. This is the first job of a set of jobs.
Synchronize with a job in which sync_count is set.
This job can be executed after all the jobs in the argument have been begun.
This job can be executed after all the jobs in the argument have terminated without errors.
This job can be executed after all the jobs in the argument have terminated with errors.
This job can be executed after all the jobs in the argument have terminated regardless of errors.
This job can be executed after (number) before dependencies (below) have been satisfied.
Jobs jobid, ... can be scheduled for execution after the current job begins.
Jobs jobid, ... can be scheduled for execution after the current job terminates without errors.
Jobs jobid, ... can be scheduled for execution after the current job terminates with errors.
Jobs jobid, ... can be scheduled for execution after the current job terminates regardless of errors.
Pass ``string'' to the target queue submission command line just before the output script name. This is useful if qprep does not implement functionality of the target system that you require, and if your local site administrator failed to include that option in the local configuration module.
You should avoid using this option in a script since it makes a script site- or machine-dependent, which misses the point of using qprep. If you find that you require this option then qprep probably needs modification. In that case, send a request for enhancement to the PST developers by sending email to ``pst-dev@pstoolkit.nwra.com''.
Pass ``string'' to the output script, preceded by the target directive prefix. This is useful if qprep does not implement functionality of the target system that you require, and if your local site administrator failed to include that option in the local configuration module.
You should avoid using this option in a script since it makes a script site- or machine-dependent, which misses the point of using qprep. If you find that you require this option then qprep probably needs modification. In that case, send a request for enhancement to the PST developers by sending email to ``pst-dev@pstoolkit.nwra.com''.
These parameters control how qprep translates directives to the target format. These are set by the local configuration module. Normally, users should not alter these values, and these directives should not occur in user qprep scripts. These directives do not typically translate directly into anything in the target script. This manual page contains these directives for completeness and to document some of the state variables that go into a local configuration module.
Set the qprep directive prefix to ``string''. By default this is ``#PSTQ'' but this assumes that the script language uses a ``#'' character as a comment character, or at least that the script language ignores strings that have a character in the first line. It is conceivable, though, that the target queue system could accept scripts in languages for which this is not true, in which case the user might have to change the source directive prefix.
Set the target script comment prefix to ``string''. By default this is ``#'', which indicates a comment in typical script languages.
Set the target script comment suffix to ``string''. By default this is ``'', (an empty string) since typical script languages have on comment terminator. Some languages (such as C) require that comments be terminated by a special string.
Set the target script directive prefix to ``string''. This value is set in the local configuration module for qprep, and its value depends on the target queue system. For example, for PBS this could be ``#PBS'' or ``#PBS -'', and for LSF this could be ``#BSUB'' or ``#BSUB -''.
User qprep scripts should NOT use this directive.
Set the target script directive separator to ``string''. This value is set in the local configuration module for qprep, and its value depends on the target queue system. For example, for PBS and LSF this could be ``'', an empty string.
Set the target queue submission command to ``string''. This value is set in the local configuration module for qprep, and its value depends on the target queue system. For example, for PBS this is probably ``qsub'', and for LSF this is probably ``bsub''. Qprep issues this command after writing the output script. The command arguments are those set by the target command line arguments directive (which you should avoid using) followed by the name of the output script file, i.e.:
To translate, but not submit, job script 'script' as a 100 node job to queue 'exqueue':
qprep nodes=100 queue=exqueue script
To translate and submit 'script' and have it send mail to user1 and user2 when the job ends:
qprep mail=e mail_to=user1,user2 submit=y script
To insert a directive (-option) that will be interpreted by the target queueing system:
qprep target_cmd_line_args='-option' submit=y script
Any of these options may instead be placed inside script such as:
#PSTQ nodes=100
#PSTQ queue=exqueue
#PSTQ mail=e
#PSTQ mail_to=user1,user2
#PSTQ target_cmd_line_args='-option'
#PSTQ submit=y
In such a case, qprep is simply invoked as:
qprep script
qprep is a script-editing translation tool in the TUSC layer of the Practical Supercomputing Toolkit (PST). By using qprep (and other elements of PST), users develop scripts and techniques that work, without modification, at all institutions supported by PST.
A script written for qprep will contain both a preamble and a body. qprep translates the preamble only. PST also contains a number of other programs to abstract particular elements of the script body, such as the command to run jobs (e.g. pstrun). Currently, the only such command that the core PST team has funding to implement is pstrun, but others are possible as future projects.
This page represents an early attempt to standardize the relevant options for qprep. If there are options that you will require in order for qprep to be useful to you, that are not listed above, please send a request for enhancement to pst-dev@pstoolkit.nwra.com describing the problem.
When a value is shown above as a set of comma-delimeted arguments, one or more of those arguments may be given as the value of the command. For instance, valid specifications of keep would be:
keep=stderr
keep=stdout
keep=stdout,stderr
keep=stderr,stdout
Possible specifications for mail would include
mail=a
mail=a,b
mail=b,e
and so on.
pstrun: a unified command-line method of running executables.
Written and Copyright (C) 2001 by Michael Gourlay, Joseph Werne, and Chris Bizon.
Documentation v0.1 assembled Friday, December 5, 2001.