#!/usr/bin/perl

__END__

=pod

=head1 NAME

B<qprep> - Translate job-submission scripts from a platform independent
format to the local, platform-dependent format.

=head1 SYNOPSIS

S<B<qprep> [key1=value1[,key2=value2 ...]] F<filename> >

=head1 DESCRIPTION

B<qprep> provides users with a uniform, site-independent command 
interface to the local queuing system, eliminating the need 
to learn the myriad syntactical variations that appear at different 
institutions to perform essentially identical job-submission tasks.

B<qprep> translates a script from a site-independent format (described 
below) into a site-dependent format, typically one using NQS (Network
Queuing System), PBS (Portable Batch System), or 
a similar queuing system.  The translated script is written to a new file,
and may optionally be submitted to a queue.   Scripts written to the B<qprep> 
standard will be usable at any site where B<qprep> is implemented, regardless
of the underlying queuing system.

=head1 OPTIONS

Options to B<qprep> are given in the form of key-value pairs.  These may
be given either at the command line, seperated by commas without whitespace,
or in the preamble to a job script. 

Job scripts contain a preamble, similar to those found in native
queuing systems.  The preamble contains directives that control
the resource limits and other attributes of the job.  B<qprep> directives
in the preamble are in the form:

#PSTQ key=value

The # defines the line as a script comment, the string PSTQ alerts B<qprep>
that the line is a directive that it needs to process, and the actual
directive is in the form of a key=value pair, whose possible values are
described below.

=head2 JOB ENVIRONMENT DIRECTIVES

These directives control the environment in which the script generated by
B<qprep> operates.

=over 4

=item B<account=account_string>

Charge the job to account B<account_string>.

=item B<username=userlist>

Run the job as the specified user. The userlist has the format:

user[@host][,user[@host], ...]

This allows the job to run as different users depending on the host
that runs the job.  Note, B<userlist> is specified with no white
space.

=item B<exportvars=y>

If B<exportvars> is set to the string "y", environment variables are exported 
from the shell in which B<qprep> is run to the script's environment.

=item B<shell=shell_name>

Define the shell that will be used to run the script.

=back

=head2 REPORTING DIRECTIVES

These directives control stdout, stderr, and mail reporting for the job.

=over 4

=item B<stderr>=F<file>

Define the F<file> to which stderr is written.  This may be the same as
the filename for stdout (see below).

=item B<stdout>=F<file>

Define the F<file> to which stdout is written.  This may be the same as
the filename for stderr.

=item B<keep=stderr,stdout>

Keep either stdout or stderr, or both, on the host where the script was
executed.  See COMMAND NOTES for a comment on the comma notation used above.

=item B<mail=a,b,e,r,t>

Send mail at different occurrences.  The user may specify multiple triggers
separated by commas.  See COMMAND NOTES for a discussion of the argument 
notation.

    a: send mail when a job is aborted
    b: send mail when a job begins execution
    e: send mail when a job ends
    r: send mail when a job is rerun
    t: send mail when a job is routed

=item B<mailto=user_list>

Specifies a comma-separated list (no white space) of users to whom mail
is sent.  Events that cause mail to be sent are set with mail=key.

=back

=head2 JOB CONTROL DIRECTIVES

These items specify particulars of how and where the job is run.

=over 4

=item B<queue=name>

Specifies the queue to which the job is submitted.

=item B<jobname=name>

Specifies a name to associate with the job.

=item B<checkpoint=(n|s|number)>

Defines the checkpointing characteristics of the job.

    n: never checkpoint
    s: checkpoint when the queue is shut down
    number: checkpoint every (number) minutes.

NOTE: Not all platforms support checkpointing.

=back

=head2 QPREP CONTROL DIRECTIVES

These items specify the behavior of qprep upon translating a script.

=over 4

=item B<silent=y>

Do not write the job identifier to stdout when the job is submitted to
a queue.

=item B<outscript>=F<outfile>

Write the translated script to F<outfile>.  If this option is not 
specified, the translated script is written to F<filename>.pst where 
F<filename> is the script argument to B<qprep>.

=item B<submit=y>

After the script has been translated, submit the new script to the queue.

=item B<erase=y>

Erase the translated script once it has been submitted to the queue.  This
action is only taken if used in conjunction with submit=y.

=back

=head2 LIMIT DIRECTIVES

These items specify resource limits.  All times are in the format
hours:minutes, where hours and minutes are numbers.  All sizes are in MB.

Limits can be specified per-process (a single process within the script)
or per-request (the total of all the processes within a script).

=over 4

=item B<nodes=number>

Specifies the number of nodes that are simultaneously required by the script.
Default is nodes=1.

=item B<cpuspernode=number>

Specifies the number of cpus per node that are simultaneously required 
by the script.  Default is cpuspernode=1.

=item B<walltime=time>

Specifies the maximum walltime to be allotted for the script execution.  

=item B<processcputime=time>

Specifies the maximum CPU time used by any single process within the script.

=item B<totalcputime=time>

Specifies the total CPU time used by all single processes within the script.

=item B<procfilesize=size>

Specifies the maximum total size of files written by a single process within
the script.

=item B<totalfilesize=size>

Specifies the maximum total size of all files written by all processes within
a script.

=item B<tapemt(a|b|c|d|e|f|g|h)=number>

This set of keys (tapemta, tapemtb, ...) specifies the maximum number of tape
drives required in the corresponding device class (a,b, ...).  Tape drive
device groups can be ascertained using the B<tpstat>(1) command.

=item B<nice=number>

Specifies the nice level at which the script is run when resources are shared.
See B<nice>(1) for a description of nice levels.

=item B<processmemory=size>

Specifies the maximum memory used by a single process within the script.
B<size> is specified in MB.  If the process does not share memory with 
other processes, this option does not need to be specified.

=item B<totalmemory=size>

Specifies the total memory used by all processes within the script. 
B<size> is specified in MB.  If the request does not share memory with
other requests, this option does not need to be specified.

=back

=head2 JOB DEPENDENCE DIRECTIVES

Sometimes it is necessary for a job to depend in some way upon another
job.  B<qprep> allows jobs to depend upon one another in three ways:

=over 4

=item 1. Multiple jobs can run at the same time using B<synccount> and 
B<syncwith>.

=item 2. Job A can execute only after job B reaches a specified point (dependency is coded in A).  This uses the B<after> commands (see below).

=item 3. When job B reaches a specified point, it allows job A to run (the dependency is coded in B).  This uses the B<before> commands (see below).

=back 

For B<after> and B<before>, dependency can be specified on jobs beginning, 
and ending (with error conditions taken into account).  This allows 
robust scripting in which different jobs run depending on the successful
or unsuccessful completion of another job.

The syntax here is very nearly that specified by PBS, which inspired these
commands.

=over 4

=item B<synccount=number>

Synchronize (B<number>) jobs.  This is the first job of a set of jobs.

=item B<syncwith=jobid>

Synchronize with a job in which B<synccount> is set.

=item B<after=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have been begun.

=item B<afterok=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
without errors.

=item B<afternotok=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
with errors. 

=item B<afterany=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
regardless of errors. 

=item B<dependson=number>

This job can be executed after (B<number>) before dependencies (below) have been
satisfied.

=item B<before=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job 
begins.

=item B<beforeok=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job 
terminates without errors.

=item B<beforenotok=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job 
terminates with errors.

=item B<beforeany=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job
terminates regardless of errors.

=back

=head2 EXTRA DIRECTIVES

=over 4

=item B<passthrough='arguments'>

Pass a quotation-enclosed set of command-line options directly to the
underlying queuing system.  This is useful if B<qprep> does not implement
functionality of the underlying system that you require.  Also prints a message
asking the user to send a request for enhancement to pst-dev@pstoolkit.org.

=back 

=head1 DIAGNOSTICS

=head1 EXAMPLES

=head1 ENVIRONMENT

=head1 FILES

=head1 NOTES

=head2 qprep'S ROLE

B<qprep> is a script-editing translation tool in the TUSC layer of the
Practical Supercomputing Toolkit (PST).  By using B<qprep> (and other
elements of PST), users develop scripts and techniques that work,
without modification, at all institutions supported by PST.

A script written for B<qprep> will contain both a preamble and a body.  
B<qprep> translates the preamble only.  PST also contains a number of other
programs to abstract particular elements of the script body, such as the
command to run jobs (e.g. B<pstrun>).  Currently, the only such command that
the core PST team has funding to implement is B<pstrun>, but others are 
possible as future projects.

This page represents an early attempt to standardize the relevant options
for B<qprep>.  If there are options that you will require in order for B<qprep>
to be useful to you, that are not listed above, please send a request for
enhancement to pst-dev@pstoolkit.org describing the problem.

=head2 COMMAND NOTES

When a value is shown above as a set of comma-delimeted arguments, one
or more of those arguments may be given as the value of the command.
For instance, valid specifications of keep would be:

    keep=stderr
    keep=stdout
    keep=stdout,stderr
    keep=stderr,stdout

Possible specifications for mail would include

    mail=a
    mail=a,b
    mail=b,e
    and so on.

=head1 SEE ALSO

pstrun: a unified command-line method of running executables.

=head1 AUTHOR

Written and Copyright (C) 2001 by Joseph Werne, Michael Gourlay, and Chris
Bizon.  www.pstoolkit.org.

=head1 HISTORY

Documentation v0.1 assembled Friday, July 13, 2001.


=cut
