#!/usr/bin/perl

#!/bin/sh
#-*-perl-*-
# \
# exec perl $0 -- ${1+"$@"}


use constant PSTDIR => "/home/gourlay/src/PST/";
use lib PSTDIR;
# use TUSC::local;
# use TUSC::localQprep;
use TUSC::qprep;
use strict ;



my($return_value) = qprepMain(\@ARGV);

exit $return_value ;



__END__

=pod

=head1 NAME

B<qprep> - Translate job-submission scripts from a platform independent
format to the local, platform-dependent format.

=head1 SYNOPSIS

S<B<qprep> [key1=value1[ key2=value2 ...]] F<filename> >

=head1 DESCRIPTION

The goal of B<qprep> is portability of batch queue scripts.
Scripts written for B<qprep> can be used at any site where B<qprep>
is installed, regardless of the target queuing system.

B<qprep> provides users with a uniform, site-independent command
interface to the local queuing system, eliminating the need
to learn the myriad syntactical variations that appear at different
institutions to perform essentially identical job-submission tasks.

B<qprep> translates a script from a site-independent format (described
below) into a site-dependent format, typically one using NQS (Network
Queuing System), PBS (Portable Batch System), or a similar queuing system.
The translated script is written to a new file, and may optionally be
submitted to a queue.

=head1 DIRECTIVE SYNTAX

Options to B<qprep> are given in the form of key-value pairs, called
"directives".  These may be given either at the command line, seperated
by white space, or in the preamble to a job script.

Job scripts contain a preamble, similar to those found in scripts
processed by native queuing system.  The preamble contains directives that
control the resource limits and other attributes of the job.  B<qprep>
directives in the preamble have this form:

#PSTQ key=value

The # character denotes the line is a script comment, the string PSTQ
alerts B<qprep> that the line is a directive that it needs to process,
and the actual directive is in the form of a key=value pair, whose
possible values are described below.

The preamble ends with this line:

#PSTQ END_OF_PREAMBLE

Qprep considers all lines following the first line matching the above
string to follow the preamble.  Qprep simply passes lines occuring
after this directive to the output script without applying translation
or interpretation.

By default qprep will also consider the first non-blank, non-comment
line to end the preamble.  See also the 'explicit_preamble' directive.

=head2 QPREP CONTROL DIRECTIVES

These directives specify the behavior of qprep upon translating
a script.  Some of these options only make sense if they occur on the
qprep command-line.  These options are not usually explicitly translated
to the target queue system;  they simply control behaviors of qprep.

=over 4

=item B<silent=y>

Do not write the job identifier to stdout when the job is submitted to
a queue.

=item B<out_script>=F<outfile>

Write the translated script to F<outfile>.  If this option is not
specified, the translated script is written to F<filename>.pst where
F<filename> is the script argument to B<qprep>.  For this option to work
properly it must occur on the command line, not inside a script.  If
this directive occurs inside a script, it changes the contents of the
parameter "out_script" but does not change the name of the actual output
filename.  That would effectively change which script gets submitted,
if the B<submit=y> directive occurs.

=item B<submit=y>

After the script has been translated, submit the new script to the queue.

=item B<erase=[y|n]>

If the value is 'y' then erase the translated script once it has been
submitted to the queue.  This action is only taken if used in conjunction
with submit=y.

If the value is 'n' then do not erase the translated script.  This
allows the user to view the translated script, e.g., for debugging.

=item B<verbose_script=[y|Y|n]>

Include extra comments in the output script that could help the user
understand how qprep translates input to the script format of the
target queue system.  The default value is "n", which tells qprep not to
produce this auxiliary information.  The value "y" (lower case) causes
qprep to include, as comments, the original input lines.  The value "Y"
(upper case) causes qprep to include yet more information.

=item B<operation_verbosity=N>

Set the verbosity of qprep.  This is useful for debugging qprep itself,
not something a typical user would find useful.

=item B<explicit_preamble=[y|n]>

Flag indicating whether the END_OF_PREAMBLE directive is required.
If explicit_preamble is 'n' then the first non-blank, non-comment line
is considered to be the end-of-preamble.

NOTE: In the context of the preamble, qprep considers a comment line to be
a line that begins with the PST qprep comment string, which is a '#', and
it setable by the preamble_comment entry in the state table.  Note that
this could differ from the comment character for the script language.

Default value is 'n'.

=back

=head2 JOB ENVIRONMENT DIRECTIVES

These directives control the environment in which the script generated by
B<qprep> operates.

=over 4

=item B<account=account_string>

Charge the job to account B<account_string>.

=item B<username=userlist>

Run the job as the specified user. The userlist has the format:

user[@host][,user[@host], ...]

This allows the job to run as different users depending on the host
that runs the job.  Note, B<userlist> is specified with no white
space.

=item B<export_vars=y>

If B<export_vars> is set to the string "y", environment variables
are exported from the shell in which B<qprep> is run to the script's
environment.

=item B<shell=shell_name>

Define the shell that will be used to run the script.

=back

=head2 REPORTING DIRECTIVES

These directives control stdout, stderr, and mail reporting for the job.

=over 4

=item B<stderr>=F<file>

Define the F<file> to which stderr is written.  This may be the same as
the filename for stdout (see below).

=item B<stdout>=F<file>

Define the F<file> to which stdout is written.  This may be the same as
the filename for stderr.

=item B<keep=stderr,stdout>

Keep either stdout or stderr, or both, on the host where the script was
executed.  See COMMAND NOTES for a comment on the comma notation used above.

=item B<mail=a,b,e,r,t>

Send mail at different occurrences.  The user may specify multiple triggers
separated by commas.  See COMMAND NOTES for a discussion of the argument
notation.

    a: send mail when a job is aborted
    b: send mail when a job begins execution
    e: send mail when a job ends
    r: send mail when a job is rerun
    t: send mail when a job is routed

=item B<mail_to=user_list>

Specifies a comma-separated list (no white space) of users to whom mail
is sent.  Events that cause mail to be sent are set with mail=key.

=back

=head2 JOB CONTROL DIRECTIVES

These items specify particulars of how and where the job is run.

=over 4

=item B<queue=name>

Specifies the queue to which the job is submitted.

=item B<parallel_env=name>

Specifies the parallel environment to use to execute the job.

=item B<job_name=name>

Specifies a name to associate with the job.

=item B<checkpoint=(n|s|number)>

Defines the checkpointing characteristics of the job.

    n: never checkpoint
    s: checkpoint when the queue is shut down
    number: checkpoint every (number) minutes.

NOTE: Not all platforms support checkpointing.

=back

=head2 LIMIT DIRECTIVES

These items specify resource limits.  All times are in the format
hours:minutes, where hours and minutes are numbers.  All sizes are in
megabytes (MB).

Limits can be specified per-process (a single process within the script)
or per-request (the total of all the processes within a script).

=over 4

=item B<nodes=number>

Specifies the number of nodes that are simultaneously required by the script.
Default is nodes=1.

=item B<cpus_per_node=number>

Specifies the number of cpus per node that are simultaneously required
by the script.  Default is cpus_per_node=1.

=item B<wall_time=time>

Specifies the maximum wall_time to be allotted for the script execution.

=item B<process_cpu_time=time>

Specifies the maximum CPU time used by any single process within the script.

=item B<total_cpu_time=time>

Specifies the total CPU time used by all single processes within the script.

=item B<proc_file_size=size>

Specifies the maximum total size, in megabytes, of files written by a
single process within the script.

=item B<total_file_size=size>

Specifies the maximum total size, in megabytes, of all files written by
all processes within a script.

=item B<tapemt(a|b|c|d|e|f|g|h)=number>

This set of keys (tapemta, tapemtb, ...) specifies the maximum number of tape
drives required in the corresponding device class (a,b, ...).  Tape drive
device groups can be ascertained using the B<tpstat>(1) command.

=item B<nice=number>

Specifies the nice level at which the script is run when resources are shared.
See B<nice>(1) for a description of nice levels.

=item B<process_memory=size>

Specifies the maximum memory used by a single process within the script.
B<size> is specified in megabytes (MB).  If the process does not share
memory with other processes, this option does not need to be specified.

=item B<total_memory=size>

Specifies the total memory used by all processes within the script.
B<size> is specified in megabytes (MB).  If the request does not share
memory with other requests, this option does not need to be specified.

=back

=head2 JOB DEPENDENCE DIRECTIVES

Sometimes it is necessary for a job to depend in some way upon another
job.  B<qprep> allows jobs to depend upon one another in three ways:

=over 4

=item 1. Multiple jobs can run at the same time using B<sync_count> and
B<sync_with>.

=item 2. Job A can execute only after job B reaches a specified point (dependency is coded in A).  This uses the B<after> commands (see below).

=item 3. When job B reaches a specified point, it allows job A to run (the dependency is coded in B).  This uses the B<before> commands (see below).

=back

For B<after> and B<before>, dependency can be specified on jobs beginning,
and ending (with error conditions taken into account).  This allows
robust scripting in which different jobs run depending on the successful
or unsuccessful completion of another job.

The syntax here is very nearly that specified by PBS, which inspired these
commands.

=over 4

=item B<sync_count=number>

Synchronize (B<number>) jobs.  This is the first job of a set of jobs.

=item B<sync_with=jobid>

Synchronize with a job in which B<sync_count> is set.

=item B<after=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have been begun.

=item B<after_ok=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
without errors.

=item B<after_not_ok=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
with errors.

=item B<after_any=jobid[,jobid, ...]>

This job can be executed after all the jobs in the argument have terminated
regardless of errors.

=item B<depends_on=number>

This job can be executed after (B<number>) before dependencies (below) have been
satisfied.

=item B<before=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job
begins.

=item B<before_ok=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job
terminates without errors.

=item B<before_not_ok=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job
terminates with errors.

=item B<before_any=jobid[,jobid, ...]>

Jobs B<jobid, ...> can be scheduled for execution after the current job
terminates regardless of errors.

=back

=head2 EXTRA DIRECTIVES

=over 4

=item B<target_cmd_line_args=string>

Pass "string" to the target queue submission command line just before
the output script name.  This is useful if B<qprep> does not implement
functionality of the target system that you require, and if your
local site administrator failed to include that option in the local
configuration module.

You should avoid using this option in a script since it makes a script
site- or machine-dependent, which misses the point of using B<qprep>.
If you find that you require this option then B<qprep> probably needs
modification.  In that case, send a request for enhancement to the PST
developers by sending email to "pst-dev@pstoolkit.org".

=item B<target_script_directive=string>

Pass "string" to the output script, NOT preceded by the target directive
prefix.  (The value may be translated by the local configuration module,
however.)  This is useful if B<qprep> does not implement functionality of
the target system that you require, and if your local site administrator
failed to include that option in the local configuration module.

You should avoid using this option in a script since it potentially
makes a script site- or machine-dependent, which misses the point of
using B<qprep>.  If you find that you require this option then B<qprep>
probably needs modification.  In that case, send a request for enhancement
to the PST developers by sending email to "pst-dev@pstoolkit.org".

=back

=head2 TRANSLATION CONTROL PARAMETERS

These parameters control how B<qprep> translates directives to the target
format.  These are set by the local configuration module.  Normally,
users should not alter these values, and these directives should not
occur in user qprep scripts.  These directives do not typically translate
directly into anything in the target script.  This manual page contains
these directives for completeness and to document some of the state
variables that go into a local configuration module.

=over 4

=item B<source_directive_prefix=string>

Set the qprep directive prefix to "string".  By default this is "#PSTQ"
but this assumes that the script language uses a "#" character as a
comment character, or at least that the script language ignores strings
that have a character in the first line.  It is conceivable, though,
that the target queue system could accept scripts in languages for which
this is not true, in which case the user might have to change the source
directive prefix.

=item B<target_comment_prefix=string>

Set the target script comment prefix to "string".  By default this is
"#", which indicates a comment in typical script languages.

=item B<target_comment_suffix=string>

Set the target script comment suffix to "string".  By default this is
"", (an empty string) since typical script languages have on comment
terminator.  Some languages (such as C) require that comments be
terminated by a special string.

=item B<target_directive_prefix=string>

Set the target script directive prefix to "string".  This value is set
in the local configuration module for qprep, and its value depends on
the target queue system.  For example, for PBS this could be "#PBS" or
"#PBS -", and for LSF this could be "#BSUB" or "#BSUB -".

User qprep scripts should NOT use this directive.

=item B<target_key_value_separator=string>

Set the target script directive separator to "string".  This value is set
in the local configuration module for qprep, and its value depends on
the target queue system.  For example, for PBS and LSF this could be "",
an empty string.

=item B<target_submit_command=string>

Set the target queue submission command to "string".  This value is set
in the local configuration module for qprep, and its value depends on
the target queue system.  For example, for PBS this is probably "qsub",
and for LSF this is probably "bsub".  Qprep issues this command after
writing the output script.  The command arguments are those set by the
target command line arguments directive (which you should avoid using)
followed by the name of the output script file, i.e.:

=over 4
target_submit_command target_cmd_line_args out_script
=back

=back

=head1 DIAGNOSTICS

=head1 EXAMPLES

To translate, but not submit, job script 'script' as a 100 node job to
queue 'exqueue':

qprep nodes=100 queue=exqueue script

To translate and submit 'script' and have it send mail to user1 and
user2 when the job ends:

qprep mail=e mail_to=user1,user2 submit=y script

To insert a directive (-option) that will be interpreted by the target
queueing system:

qprep target_cmd_line_args='-option' submit=y script

Any of these options may instead be placed inside script such as:

#PSTQ nodes=100

#PSTQ queue=exqueue

#PSTQ mail=e

#PSTQ mail_to=user1,user2

#PSTQ target_cmd_line_args='-option'

#PSTQ submit=y

In such a case, qprep is simply invoked as:

qprep script

=head1 ENVIRONMENT

=head1 FILES

=head1 NOTES

=head2 qprep'S ROLE

B<qprep> is a script-editing translation tool in the TUSC layer of the
Practical Supercomputing Toolkit (PST).  By using B<qprep> (and other
elements of PST), users develop scripts and techniques that work,
without modification, at all institutions supported by PST.

A script written for B<qprep> will contain both a preamble and a body.
B<qprep> translates the preamble only.  PST also contains a number of other
programs to abstract particular elements of the script body, such as the
command to run jobs (e.g. B<pstrun>).  Currently, the only such command that
the core PST team has funding to implement is B<pstrun>, but others are
possible as future projects.

This page represents an early attempt to standardize the relevant options
for B<qprep>.  If there are options that you will require in order for B<qprep>
to be useful to you, that are not listed above, please send a request for
enhancement to pst-dev@pstoolkit.org describing the problem.

=head2 COMMAND NOTES

When a value is shown above as a set of comma-delimeted arguments, one
or more of those arguments may be given as the value of the command.
For instance, valid specifications of keep would be:

    keep=stderr
    keep=stdout
    keep=stdout,stderr
    keep=stderr,stdout

Possible specifications for mail would include

    mail=a
    mail=a,b
    mail=b,e
    and so on.

=head1 SEE ALSO

http://pstoolkit.org

pstrun: a unified command-line method of running executables.

=head1 AUTHOR

Written and Copyright (C) 2001 by Michael Gourlay, Joseph Werne, and Chris
Bizon.

=head1 HISTORY

Documentation v0.1 assembled Friday, December 5, 2001.


=cut
