#!/usr/bin/perl

=pod

A testing script for archive.  Besides exercising the archive script, it shows 
how archive can be used to write cross-platform scripts.   TestArchive is 
completely and utterly independent of the local architecture/operating system
as long as PST is installed (with a local.pm that supports all the operations) 
and perl is working. 

TestArchive tests the following commands:

  archive stat

  archive ls
  archive ls "fil*"

  archive mkdir dir1
  archive mkdir dir1/dir2     (dir1 doesn't exist)
  archive mkdir -p dir1/dir2  (dir1 doesn't exist)
  archive mkdir dir1 dir2
  archive mkdir -m MODE dir1
  archive mkdir -p -m MODE dir1/dir2 dir1/dir3 (dir1 doesn't exist)

  archive put -C path file1                       (file1 doesn't exist)
  archive put -C path file1
  archive put -C path -z file1
  archive put -C path dir1/file1                  (dir1 doesn't exist)
  archive put -C path -p dir1/file1               (dir1 doesn't exist)
  archive put -C path -M file1                    (fails for local.pm.test)
  archive put -C path file1 file2
  archive put -C path fil*                        (test globbing)
  archive put -C path -t tarfile.tar file1 file2
  archive put -C path -S -t tarfile.tar file1 file2
  archive put -C path -t tarfile.tar directory1
  archive put -C path -D file1
  archive put -C path -D -z file1
  archive put -C path -D -z -t tarfile.tar -M fil* (Fails for local.pm.test)

  archive get -C path file1
  archive get -C path file1 file2
  archive get -C path "fil*"                       (unglobs to file1 file2)
  archive get -C path "isa*"                       (unglobs to isafile isadir)
  archive get -C path file1                        (path doesn't exist)
  archive get -C path file1                        (file1 doesn't exist)
  archive get -C path -x file.tar
  archive get -C path -x -S file.tar
  archive get -C path -x file.tgz
  archive get -C path -x "*gz"     
  archive get -C path -x -z "*gz"
  archive get -C path -M file                      (Fails for local.pm.test!)

It does not test:

  -retry options
  -asf host options
  -n (not implemented in archive)
  -f force options (not implemented yet)

Archive also does not test its link-handling facilities: the difficulty is in
setting up the test cases.  Because archive doesn't have the ability to
create links on the mass-store, their creation would have to be site 
specific.  There are a couple ways around this: we could have a site-specific
script that TestArchive uses to set up the links.  We could add the ability
to create links to archive. 

There is an inherent difficulty in testing archive ls along with the other
commands.   We need to use those commands to create somthing for archive ls
to look at (to test archive ls), and we need to use archive ls to look at
the results of archive mkdir and put, to test mkdir and put commands.  We
essentially ignore this difficulty.  It would take a fairly special 
circumstance to make everything look as if it worked when it didn't.  The
problems are more likely to arise when trying to diagnose a failure of 
TestArchive.  Is the failure due to a failure in the archive get,put,etc
or due to a failure of archive ls?  The user will have to resort to non-pst
commands to sort that out.  This is the reason that archive ls reports
success multiple times --- as it gets tested by mkdir/get/put we test out
various ls options as well.


If you want to be absolutely sure that everything worked, look (using non
pst commands) for this directory structure on your mass-store:

  $roottestdir/$gettestdir/gztarfile.tgz
  $roottestdir/$gettestdir/tarfile.tar
  $roottestdir/$gettestdir/testfile1.plain
  $roottestdir/$gettestdir/testfile2.plain
  $roottestdir/$gettestdir/testfile3.gz
  $roottestdir/$mkdirtestdir/mkdir_p_parent/mkdir_p_child
  $roottestdir/$mkdirtestdir/testmode/
  $roottestdir/$mkdirtestdir/test_multi1/
  $roottestdir/$mkdirtestdir/test_multi2/
  $roottestdir/$mkdirtestdir/test_options/options1
  $roottestdir/$mkdirtestdir/test_options/options2
  $roottestdir/$puttestdir/dashC/dashC.file
  $roottestdir/$puttestdir/test_multi/PSTtest1.file
  $roottestdir/$puttestdir/test_multi/PSTtest2.file
  $roottestdir/$puttestdir/test_p/test_p.file
  $roottestdir/$puttestdir/test_put_delete/filetodelete
  $roottestdir/$puttestdir/test_put_glob/PSTglobtest.file1
  $roottestdir/$puttestdir/test_put_glob/PSTglobtest.file2
  $roottestdir/$puttestdir/test_put_zip_and_delete/filetozandd.gz
  $roottestdir/$puttestdir/test_tar/PSTtest.tar
  $roottestdir/$puttestdir/test_tar_dir/PSTtest_dir.tar
  $roottestdir/$puttestdir/test_tar_S/PSTtest_S.tar
  $roottestdir/$puttestdir/test_zip/file_to_zip.gz

Note that local.pm.test will fail if the migration commands are checked.
(Since migration won't be supported for the local.pm.test).  Therefore, they
are commented out in this version of TestArchive.    If you want to test your
own version of TestArchive, you should uncomment these lines.

The previous version of TestArchive required that archive ls -l returned
mode and migration information.  This has been supplanted by local.pm 
subroutines site_get_file_mode and site_file_is_migrated.  However, these
local routines are not included in the archive interface.  So here we
'use TUSC::local' and call these subs directly.   In order to do so, we have
to do a little extra processing that archive would normally do for us, such
as getting the user, host, and remote home directory.   If we make these
functions part of archive, perhaps of an archive stat sort of command, we
can remove the direct local.pm calls from TestArchive, which will clean up
several subroutines.


[CB]

=cut

use strict;
use English;     #lets us use $CHILD_ERROR instead of $?.
use TUSC::local; #so that we can check the mode.
use File::Basename;


#the directories into which TestArchive places files on the mass store:
my $roottestdir  = 'PSTTestDirectory1'; 
my $mkdirtestdir = $roottestdir."/mkdir_test"; 
my $puttestdir   = $roottestdir."/put_test"; 
my $gettestdir   = $roottestdir."/get_test"; 

print_warning_message();

#
# Do tests.
#  Test archive stat
#  Test archive ls
#  Test archive mkdir
#  Test archive put
#  Test archive get
#


#
# Test archive stat 
#

test_stat();
print "archive stat  => passed\n";

#
# Test archive ls (initial) 
#
#  We want to do these first, since we can then use archive ls to make
#  sure that the other commands work well. We do so by checking 
#  for $roottestdir.
#

check_for_roottestdir($roottestdir);
print "archive ls    => passed (initial)\n";

#
# Test archive mkdir 
#

print "archive mkdir ";

test_mkdir_no_options($roottestdir);

print "=";

#if that worked, lets make the rest of the sub-directories we want
`./archive mkdir $mkdirtestdir 2>/dev/null`;
`./archive mkdir $puttestdir 2>/dev/null`;
`./archive mkdir $gettestdir 2>/dev/null`;

#back to the mkdir tests:
test_mkdir_p($mkdirtestdir);
print "=";
test_mkdir_multi($mkdirtestdir);
print "=";
test_mkdir_mode($mkdirtestdir);
print "=";
test_mkdir_options($mkdirtestdir);
print "=> passed\n";


#
# archive put
#

# Commented out ones fail in local. Uncomment once we get the bugs worked out.

print "archive put ";
test_put_no_local_file($puttestdir);  
print "=";
test_put_dashC($puttestdir);
print "=";
test_put_zip($puttestdir);   
print "=";
test_put_p($puttestdir);       
print "=";
test_put_migrate($puttestdir);       
print "=";
test_put_multi($puttestdir);
print "=";
test_put_glob($puttestdir);
print "=";
test_put_tar($puttestdir);
print "=";
test_put_tar_S($puttestdir);
print "=";
test_put_tar_directory($puttestdir);
print "=";
test_put_delete_local_file($puttestdir);
print "=";
test_put_zip_and_delete($puttestdir);
print "=";
test_put_options($puttestdir);        
print "=> passed\n";

#
# archive get
#

# prepare several files on the mass store that'll be used in the get tests:
#  $gettestdir/testfile1.plain
#  $gettestdir/testfile2.plain
#  $gettestdir/testfile3.gz
#  $gettestdir/tarfile.tar
#  $gettestdir/gztarfile.tgz
#  $gettestdir/isadir/
#  $gettestdir/isafile

#also test globbing in ls, since these look for *.plain, *.tgz, etc:
init_get_tests($gettestdir);
print "archive get ";
test_get_dashC($gettestdir);
print "=";
test_get_multi($gettestdir);
print "=";
test_get_glob($gettestdir);
print "=";
test_get_glob_to_file_and_dir($gettestdir);
print "=";
test_get_no_such_path($gettestdir);
print "=";
test_get_no_such_file($gettestdir);
print "=";
test_get_tar($gettestdir);
print "=";
test_get_tar_S($gettestdir);
print "=";
test_get_gz($gettestdir);
print "=";
test_get_tgz($gettestdir);
print "=";
test_get_stargz($gettestdir);
print "=";
test_get_and_unzip_stargz($gettestdir);
print "=";
test_get_migrate($gettestdir); 
print "=> passed\n";

#
# all tests complete
#


print <<end_message

All tests successfully completed.

There is a (slim) possiblity that a strange confluence of bugs in archive
put, archive ls, archive mkdir, and archive get has lead to incorrect passing
of the tests (since we use archive commands to test the results of other
archive commands). If you want to be 100% sure that everything worked, compare
the directory structure on your mass-store to that listed in the pod comment at
the beginning of TestArchive.pl.

Before you run TestArchive again, you must remove the files left on your
mass-store.

end_message
;

#
#
# Subroutines: Each test is a separate sub.
#
#

sub print_warning_message{

  #
  # Print the following warning for the user. It describes problems they
  # may encounter when using TestArchive
  #

  my ($warning_message);

  $warning_message = <<end_message; 
This script will now test the local installation of archive by attempting
to run a number of archive commands.  The script will exit at the first
failed test.

Unfortunately, archive rm and archive rmdir are not yet implemented.  
This test script cant clean up after itself, and will leave some garbage 
on your mass-store directory.   Please remove the directory TestDirectory1
and all files/directories below it. If you do not, there will be problems in
rerunning the test script.

Also, please make sure that PSTTestDirectory1 doesnt already exist in 
your home directory on the mass-store.  We use archive to check, but
since the purpose of this script is to test archive, it is better to pre-test
by hand.

---------------------------------------------------------------------------
end_message
;

  print "\n$warning_message\n";

}




sub test_stat
{

  #
  # archive stat test
  # find out whether archive is up.
  # This one's a little tricky to really test.  All we're checking is the
  # error condition.  It doesn't guarantee that the ASF is actually up
  # through an independent check (it'd be site specific).
  #


  `./archive stat`;                           #archive stat returns 0 if up.

  $CHILD_ERROR == 0 || die("Error in archive stat");  #if ASF not up, die.

}




sub check_for_roottestdir{

  #
  #  This does double duty as a check on archive ls and a check to make sure
  #  that the junk left over from the script doesn't already exist on the
  #  mass-store (which will mess up the results of TestArchive).
  #
  
  my ($testdir) = @_;      #root testing directory
  my ($output);            #stdout from archive ls. 
  my ($errorstring);

  $output = `./archive ls`;
  $CHILD_ERROR == 0 || die("archive ls returned error");

  $errorstring = <<end_message

  There is already a $testdir on the mass store. 
  You have not cleaned up from a previous test. 
  Please do so before trying to run TestArchive 
  (On the plus side, archive ls is working!)

end_message
  ;

  #check the output of ls to see whether the PST test directory is already
  #there.  If it is, ask the user to clean it up and die.
  if ($output =~ /$testdir/){
    die($errorstring);
  }

}





sub test_mkdir_no_options{

  #
  #  Test mkdir without any options.
  #  This is where the root testing directory and other subdirs gets made.
  #

  my ($testdir) = @_;     #root testing directory
  my ($lsoutput);         #output of archive ls

  `./archive mkdir $testdir 2>/dev/null`;
  ($CHILD_ERROR == 0) || 
     die ("mkdir (no options) returned nonzero exit status.");

  #was directory created?
  $lsoutput = `./archive ls`;
  ($lsoutput =~ /$testdir/) || 
   die ("mkdir failed to create the remote directory $testdir, or ls failed to find it once created");

}






sub test_mkdir_p{

  #
  # Exercise archive mkdir -p
  #  Try making a recursive directory without -p (which fails)
  #  Try making a recursive directory with -p (which succeeds)
  # We'll try to make $testdir/$parentdir/$childdir:
  #

  my ($testdir)=@_;      #root testing dir
  my ($lsoutput);        #archive ls output
  my ($parentdir) = "mkdir_p_parent"; 
  my ($childdir) = "mkdir_p_child";


  #
  # 1. Test whether recursive mkdir happens without -p (it shouldn't):
  #
  `./archive mkdir $testdir/$parentdir/$childdir 2>/dev/null`;
  ($CHILD_ERROR != 0) || die ("mkdir with a non-existent parent directory returned with an error code of zero");

  #was directory created?
  $lsoutput = `./archive ls $testdir`;
  ($lsoutput !~ /$parentdir/)  ||
      die("mkdir created a parent directory, even without the -p option.\n"); 


  # 
  # 2. Test whether recursive mkdir happens with -p option (it should):
  #
  `./archive mkdir -p $testdir/$parentdir/$childdir 2>/dev/null`;
  ($CHILD_ERROR == 0) || die ("mkdir -p returned nonzero exit status");

  #was directory created?
  $lsoutput = `./archive ls $testdir/$parentdir`;
  ($lsoutput =~ /$childdir/) || 
    die("mkdir -p failed to create child directory\n"); 

}





sub test_mkdir_multi{

  #
  #  Test multiple directory creation by archive mkdir
  #

  my ($testdir) = @_;      #root dir for tests
  my ($lsoutput);          #output from archive ls
  my ($directory1) = "test_multi_1";
  my ($directory2) = "test_multi_2";

  # Test multiple dir creation
  `./archive mkdir $testdir/$directory1 $testdir/$directory2 2>/dev/null`;
  ($CHILD_ERROR == 0) || 
      die ("mkdir of multiple directories returned nonzero exit status");

  #were directories created?
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$directory1/  || die ("$directory1 not created");
  $lsoutput =~ /$directory2/  || die ("$directory2 not created");

}




sub test_mkdir_mode{

  #
  # Test the -m mode option of archive mkdir by creating a dir with mode 700.
  #


  my ($testdir) = @_;    #root directory for tests.
  my ($output);          #output of archive ls
  my ($errorstring);     #gets printed on failure.
  my ($directory) = "test_mode";
  my (@hostlist,$setofhosts);
  my ($user,$host,$home,$mode);


  #Test mode creation
  `./archive mkdir -m 700 $testdir/$directory 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("archive -m MODE returned nonzero exit status");

  $output = `./archive ls $testdir`;
  $output =~ /$directory/ || die ("$directory not created when -m used");

  #because we are using the local.pm routine site_get_file_mode directly,
  # we have to set a few variables that archive would normally set for us.
  $user = $ENV{'USER'};

  $setofhosts = site_get_archive_host($user);
  @hostlist=split(",",$setofhosts);
  $host = $hostlist[0];

  $home = site_get_archive_home($user);
  $mode = site_get_file_mode($user,$host,"$home/$testdir/$directory");
  (0700 == $mode) || die ("$directory created with the wrong mode");

}






sub test_mkdir_options{


  my ($testdir) = @_;          #root for tests 
  my ($lsoutput);              #output for archive ls
  my (@lslines);               #output for ls seperated into lines
  my ($lsfile);                #an element of @lslines
  my ($ncreated);              #num of created files with the right mode
  my ($errorstring);           #print when error encountered.
  my ($parentdirectory) = "test_options";       
  my ($commonstring)    = "options";
  my ($childdir1)       = $commonstring."1";
  my ($childdir2)       = $commonstring."2";
  my ($direc);
  my ($user,$host,$home,$mode);
  my ($setofhosts,@hostlist);
   
  $errorstring = "mkdir test with -p and -m and multiple directories failed\n";
 
  #
  # Test recursive (-p), mode setting (-m mode) and multiple directory
  #  creation simultaneously.
  #

  `./archive mkdir -p -m 700 $testdir/$parentdirectory/$childdir1 $testdir/$parentdirectory/$childdir2 2>/dev/null`;
  $CHILD_ERROR == 0 || 
         die ("archive mkdir -p -m dir1 dir2 returned nonzero status");

  # get the output from ls, split it into lines.
  $lsoutput = `./archive ls $testdir/$parentdirectory`;
  @lslines = split("\n",$lsoutput);

  #were the directories created ok?
  ( $lsoutput =~ /$childdir1/ && $lsoutput =~ /$childdir2/  ) ||
    die($errorstring);

  #because we are using the local.pm routine site_get_file_mode directly,
  # we have to set a few variables that archive would normally set for us.
  $user = $ENV{'USER'};

  $setofhosts = site_get_archive_host($user);
  @hostlist=split(",",$setofhosts);
  $host = $hostlist[0];

  $home = site_get_archive_home($user);

  #make sure the modes are right
  $direc = "$home/$testdir/$parentdirectory";
  $mode = site_get_file_mode($user,$host,"$direc/$childdir1");
  (0700 == $mode) || die ("$childdir1 created with the wrong mode");
  $mode = site_get_file_mode($user,$host,"$direc/$childdir2");
  (0700 == $mode) || die ("$childdir1 created with the wrong mode");


}



sub make_test_files{

  #
  # Make the files to be used in testing put/get.
  #  Given a list of filenames, create any files that don't already exist.
  #

  my (@createlist) = @_; 
  my $filehandle;
  my $filename;

  foreach $filename (@createlist){
    if (!(-e $filename)){
      open  ($filehandle,">$filename");
      print $filehandle "This space intentionally left blank" ;
      close ($filehandle);
    }
  }
  
}



sub test_put_dashC{

  #
  #  Test putting a file on the mass store with only the -C option
  #

  my ($testdir) = @_;  #root testing directory, test file
  my ($lsoutput);      #output of archive ls
  my ($subdir) = "dashC";
  my ($testfile) = "dashC.file";

  make_test_files($testfile);

  #set up the subdirectory
  $testdir = $testdir."/".$subdir;
  `./archive mkdir $testdir 2>/dev/null`;

  #do the put
  `./archive put -C $testdir $testfile `;
  $CHILD_ERROR == 0 || die ("archive put -C returned with nonzero exit status");

  #was the file put?
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$testfile/ || die ("put -C failed");

  unlink ($testfile);

}




sub test_put_multi{

  #
  #  Test putting multiple (2) files on the mass-store.
  #

  my ($testdir) = @_;        #root testing directory, test files
  my ($lsoutput);            #output of ls
  my ($testfile1) = "PSTtest1.file";
  my ($testfile2) = "PSTtest2.file";
  my ($subdir)    = "/test_multi";
  $testdir = $testdir.$subdir;

  #make the remote directory and local files.
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1,$testfile2);

  `./archive put -C $testdir $testfile1 $testfile2 2>/dev/null`;
  $CHILD_ERROR == 0 || 
     die("put with multiple files returned nonzero exit status");

  #were the files put?
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$testfile1/ || 
     die ("put with multiple files failed: first file ($testfile1) not put");
  $lsoutput=~ /$testfile2/ || 
     die ("put with multiple files failed: second file ($testfile2) not put");

  unlink $testfile1;
  unlink $testfile2;

}


 

sub test_put_p{

  #
  #  Test putting a file on the mass store with the -p option.
  #  Test by trying with and without the -p and making sure it only
  #  works with -p
  #

  my ($testdir) = @_;              #root testing directory
  my ($lsoutput);                  #output of ls
  my ($testfile) = "test_p.file";
  my ($subdir)   = "/test_p";      #the subdirectory to make and put into

  make_test_files($testfile);

  # Should fail without -p (& return nonzero exit status).
  `./archive put -C $testdir.$subdir $testfile 2>/dev/null`;
  $CHILD_ERROR != 0 || die ("archive without -p returned with exit status 0");


  #if it didn't work, then subdir shouldn't exist and won't show up on ls.
  $lsoutput = `./archive ls $testdir`;
  ($lsoutput !~ /$subdir/)  ||
      die("-p option to mkdir not needed when it should be needed.\n"); 


  # 
  # Test whether put works with -p option (it should):
  #
  `./archive put -p -C $testdir$subdir $testfile 2>/dev/null`;
  $CHILD_ERROR==0 || die ("put -p returned nonzero exit status");

  #find out whether file was put (it should be).
  $lsoutput = `./archive ls $testdir$subdir`;
  ($lsoutput =~ /$testfile/) || die("put -p failed \n"); 

  unlink $testfile;

}




sub test_put_tar{

  #
  #  Test the -t option of put (make a tarfile and put it).
  #

  my ($testdir) = @_;                #root testing directory, test files
  my ($subdir) = "/test_tar";        #subdir where we'll put the tarfile
  my ($lsoutput);                    #output of ls
  my ($tarfile) = "PSTtest.tar";     #name of the tarfile to make/put
  my ($testfile1) = "tartest.file1"; #files to put in the tarfile:
  my ($testfile2) = "tartest.file2";

  #set up remote dir, local test files.
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1,$testfile2);

  #do the put -t
  `./archive put -C $testdir -t $tarfile $testfile1 $testfile2 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("archive put -t returned nonzero exit status");

  #test whether file was put, and local tarfile removed.
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$tarfile/  || die ("put -t didn't put $tarfile");
  !(-e $tarfile) || die ("put -t didn't delete local tarfile");

  #clean up
  unlink $testfile1;
  unlink $testfile2;


}




sub test_put_tar_S{

  #
  #  Test the -t option of put (make a tarfile and put it) along with -S.
  #   (don't remove local copy
  #

  my ($testdir) = @_;                 #root testing directory
  my ($lsoutput);                     #output of archive ls
  my ($subdir) = "/test_tar_S";       #remote subdir where tarfile goes
  my ($tarfile) = "PSTtest_s.tar";    #name of the tarfile to make/put
  my ($testfile1) = "tartest.file1";  #files in the tarfile:
  my ($testfile2) = "tartest.file2";

  #make remote directory, local files
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1,$testfile2);

  #put -t -S
  `./archive put -C $testdir -t $tarfile -S $testfile1 $testfile2 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("put -t -S returned nonzero exit status");

  #check whether tarfile put and not removed locally
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$tarfile/ || die ("put -t -S didn't put tarfile");
  (-e $tarfile) || die ("put -t -S removed local tarfile");

  #clean up
  unlink $testfile1;
  unlink $testfile2;
  unlink $tarfile;

}




sub test_put_tar_directory{

  #
  #  Test the -t option of put (make a tarfile and put it) by tarring a
  #   whole directory
  #

  my ($testdir) = @_;                #root testing directory
  my ($subdir) = "/test_tar_dir";    #remote subdirectory where tar file goes
  my ($tardir) = "tar_dir";          #local directory that gets tarred.
  my ($lsoutput);                    #output of ls
  my ($tarfile) = "PSTtest_dir.tar"; #name of the tarfile to make/put
  my ($testfile1) = "tartest.file1"; #files int he tarfile
  my ($testfile2) = "tartest.file2"; #files int he tarfile

  #make remote directory
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;

  #make local directory, files
  ###printf "mkdir $tardir \n" ;

  mkdir $tardir ;
  make_test_files("$tardir/$testfile1","$tardir/$testfile2");

  #put -t directory
  `./archive put -C $testdir -t $tarfile $tardir 2>/dev/null`;
  $CHILD_ERROR==0 || die ("put -t on a directory returned nonzero exit status");

  #check to see that tarfile was put and locally removed
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$tarfile/ || 
    die ("put -t directory did not put $tarfile on mass store");
  !(-e $tarfile)  || 
    die ("put -t directory did not remove local $tarfile");

  #clean up locally
  unlink "$tardir/$testfile1";
  unlink "$tardir/$testfile2";
  rmdir $tardir;

}






sub test_put_migrate{

  #
  #  Test the -M option of put (put a file and migrate it).
  #

  my ($testdir) = @_;               #root testing directory
  my ($subdir) = "/test_migrate";   #remote directory where file is put
  my ($lsoutput);                   #output of ls 
  my ($testfile) = "migrated.file"; #test file
  my ($user,$home,$host,$migrated);
  my ($setofhosts,@hostlist);
  
  #make remote directory, local test file
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile);

  #archive with migration
  `./archive put -C $testdir -M $testfile 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("archive put -M returned nonzero exit status");

  #did file get transferred?
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$testfile/ || die ("put -M didn't put $testfile");

  #because we are using the local.pm routine site_get_file_mode directly,
  # we have to set a few variables that archive would normally set for us.
  $user = $ENV{'USER'};

  $setofhosts = site_get_archive_host($user);
  @hostlist=split(",",$setofhosts);
  $home = site_get_archive_home($user);

  #was file migrated?
  $migrated = site_file_is_migrated($user,$host,"$home/$testdir/$testfile");
  $migrated || die("$testfile not migrated");

  #clean up
  unlink $testfile;

}





sub test_put_glob{

  #
  #  Test the file globbing of put (i.e. using *).
  #

  
  my ($testdir) = @_;                     #root testing directory
  my ($lsoutput);                         #output of ls
  my ($subdir) = "/test_put_glob";        #remote subdir where files go
  my ($testprefix) = "PSTglobtest";       #test files:
  my ($testfile1) = $testprefix.".file1"; 
  my ($testfile2) = $testprefix.".file2";

  #create remote directory, local files
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1); 
  make_test_files($testfile2); 

  #put "test*"
  `./archive put -C $testdir $testprefix* 2>/dev/null`;
  $CHILD_ERROR == 0 || 
      die ("test \"$testprefix*\" returned nonzero exit status");

  #see whether files were put on mass store
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$testfile1/ ||
     die ("put with file glob failed: $testfile1 not put on mass store");
  $lsoutput =~ /$testfile2/ || 
     die ("put with file glob failed: $testfile2 not put on mass store");

  #clean up
  unlink $testfile1;
  unlink $testfile2;

}





sub test_put_no_local_file{

  #
  # Make sure that put acts gracefully when a nonexistant local file is 
  # given to it.  It should die and return a nonzero exit status.
  #

  my ($testdir) = @_;                #root testing directory
  my ($lsoutput);                    #output of archive ls
  my ($nosuchfile) = "PSTBogusFile"; #filename that doesn't exist in local. 

  #make sure that our nonexistent file really doesn't exist. Keep adding
  #'x' to the end of the filename until we get one that doesn't exist.
  while ( -e $nosuchfile ) {
    $nosuchfile = $nosuchfile."x";
  }

  #try putting a file that doesn't exist
  `./archive put -C $testdir $nosuchfile 2>/dev/null`;
  $CHILD_ERROR != 0 || 
   die ("archive put on a nonexistent file returned a zero exit status");

  #make sure nobody manufactured the file!
  $lsoutput = `./archive ls $testdir`;
  ($lsoutput !~ /$nosuchfile/) || 
     die ("archive put somehow put a nonexistent file.");


}





sub test_put_zip_and_delete{

  #
  #  Test put delete (-D) with zip (-z)
  #    Neither zip nor original file should exist at end of test.
  #

  my ($testdir) = @_;                       #root testing directory
  my ($lsoutput);                           #output of ls
  my ($subdir) = "/test_put_zip_and_delete";#remote subdir where zip file goes
  my ($testfile1) = "filetozandd";          #test file
  my ($zipfile);                            #zipped test file 
  
  #make remote directory and local files
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1);

  #put -z -D
  `./archive put -C $testdir -z -D $testfile1 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("put -D -z returned non-zero exit status");


  #Look for zipfile on remote space
  $lsoutput = `./archive ls $testdir`;
  $zipfile = "$testfile1.gz";

  #tests
  $lsoutput =~ /$zipfile/ || die ("$zipfile not found in $testdir");
  !(-e $testfile1)  ||       die ("$testfile1 still exists locally");
  !(-e $zipfile )   ||       die ("$zipfile still exists locally");


}





sub test_put_delete_local_file{

  #
  #  Test put with delete local file (-D option)
  #

  my ($testdir) = @_;                #root testing directory
  my ($lsoutput);                    #output of ls
  my ($subdir) = "/test_put_delete"; #remote subdir where files go
  my ($testfile1) = "filetodelete";  #test file
  
  #create remote directory, local file.
  $testdir = $testdir.$subdir;
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile1);

  #put and delete
  `./archive put -C $testdir -D $testfile1 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("put -D returned nonzero exit status");     

  #check that file was put on remote dir and deleted locally.
  $lsoutput = `./archive ls $testdir`;
  $lsoutput =~ /$testfile1/ || die("put -D failed to put file on mass-store");
  !(-e $testfile1)  || die ("put -D failed to remove local file");

}





sub test_put_zip{

  #
  #  Test zip on put (-z option)
  #

  my ($testdir) = @_;                 #root testing directory
  my ($lsoutput);                     #output of ls
  my ($subdir)     = "test_zip";      #remote dir where zipfile will be put
  my ($testfile)   = "file_to_zip";   #name of test file
  my ($zippedfile) = $testfile.".gz"; #name of gzipped test file.

  #set up test remote dir, local file
  $testdir = $testdir."/".$subdir; 
  `./archive mkdir $testdir 2>/dev/null`;
  make_test_files($testfile);

  #put -z 
  `./archive put -C $testdir -z $testfile 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("put -z returned nonzero exit status");

  #check that the zip file is on the remote dir and that the original
  # (nonzipped) file remains locally.
  $lsoutput  = `./archive ls $testdir`;
  $lsoutput =~ /$zippedfile/ || die("put -z failed to put file on mass-store");
  (-e $testfile)             || die ("put -z removed local file");
  !(-e $zippedfile)          || die ("put -z left zipped file in local space");

  unlink $testfile;

}




sub test_put_options{

  #
  #  Test many put options. -C, -p, -D, -z, -t, -M and fileglobbing
  #

  #root testing directory, test file 
  my ($testdir) = @_;                 #root directory for testing
  my ($subdir) = "/full_put_test";    #remote subdir where file will be put
  my ($lsoutput);                     #output of archive ls
  my ($testfilepre) = "fullputtest";  #prefix used in file glob
  my ($testfile1) = "$testfilepre.f1";#test files
  my ($testfile2) = "$testfilepre.f2";#test files
  my ($tarfile) = "PSTputtest.tar";   #name of local tar file.
  my ($tgzfile) = "PSTputtest.tgz";   #name of remote tar file.
  my ($user,$host,$home,$migrated);   #used in checking file migration.
  my ($setofhosts,@hostlist);
  

  #set remote dir; create local test files
  $testdir = $testdir.$subdir;
  make_test_files($testfile1,$testfile2);

  #put -C -p -D -z -t -M glob
  `./archive put -C $testdir -p -D -z -t $tarfile -M $testfilepre* 2>/dev/null`;
  $CHILD_ERROR == 0 || 
     die ("put -C -p -D -z -t -M $testfilepre* returned nonzero exit status");

  #make sure that the original file gets put on mass store.
  $lsoutput  = `./archive ls $testdir`;
  $lsoutput =~ /$tgzfile/ || die ("$tgzfile not on mass store");
  #make sure local files removed.
  !(-e $testfile1) || die ("$testfile1 not removed");
  !(-e $testfile2) || die ("$testfile2 not removed");
  !(-e $tarfile)   || die ("$tarfile not removed");
  !(-e $tgzfile)   || die ("$tgzfile not removed");

  #check remote migration

  #because we are using the local.pm routine site_get_file_mode directly,
  # we have to set a few variables that archive would normally set for us.
  $user = $ENV{'USER'};

  $setofhosts = site_get_archive_host($user);
  @hostlist=split(",",$setofhosts);
  $host = $hostlist[0];

  $home = site_get_archive_home($user);
  #was file migrated?
  $migrated = site_file_is_migrated($user,$host,"$home/$testdir/$tgzfile");
  $migrated || die("$tgzfile not migrated");


}




sub init_get_tests{

  #
  #  The get tests require that certain files are on the mass-store. This
  #   subroutine creates:
  #
  #  $gettestdir/testfile1.plain
  #  $gettestdir/testfile2.plain
  #  $gettestdir/testfile3.gz
  #  $gettestdir/tarfile.tar
  #  $gettestdir/gztarfile.tgz
  #  $gettestdir/isadir/
  #  $gettestdir/isafile
  #

 
  my ($gettestdir) = @_;       # root for the get's. It should already exist.
  my ($tfile1) = "testfile1.plain";
  my ($tfile2) = "testfile2.plain";
  my ($tfile3) = "testfile3";
  my ($tarfile1)  = "tarfile.tar";
  my ($gtarfile1) = "gztarfile.tgz";
  my ($isadir) = "isadir";
  my ($isafile) = "isafile";

  make_test_files($tfile1,$tfile2,$tfile3,$isafile);

  `./archive put -C $gettestdir $tfile1 $tfile2`;
  `./archive put -C $gettestdir -D -z $tfile3`;
  `./archive put -C $gettestdir -t $tarfile1 $tfile1 $tfile2`;
  `./archive put -C $gettestdir -D -t $gtarfile1 -z $tfile1 $tfile2`;
  `./archive put -C $gettestdir -D $isafile`;
  `./archive mkdir $gettestdir/$isadir 2>/dev/null`;
  

  

}





sub test_get_dashC{


  #
  #  Test getting a file on the mass store with only the -C options
  #  This also tests the file-globbing of ls
  #

  my ($testdir) = @_;  #root testing directory
  my ($testfile) = "testfile1.plain";      #file to get

  #do the get
  `./archive get -C $testdir $testfile`;
  $CHILD_ERROR == 0 || die ("archive get -C returned nonzero exit status");     

  #test that get worked
  (-e $testfile ) || die ("get -C failed to get $testfile");

  #clean up
  unlink ($testfile);

}




sub test_get_multi{


  #
  #  Test getting multiples files on the mass store with no options
  #

  my ($testdir) = @_;            #root testing directory
  my ($testfile) = "testfile1.plain";
  my ($testfile2) = "testfile2.plain";

  #do the get
  `./archive get -C $testdir $testfile $testfile2`;
  $CHILD_ERROR == 0 || 
    die ("archive get with multiple files returned nonzero errror status");

  #check that both files were actually gotten.
  -e $testfile || die ("$testfile not returned with multiple file get");
  -e $testfile2 || die ("$testfile2 not returned with multiple file get");

  #clean up
  unlink ($testfile);
  unlink ($testfile2);

}




sub test_get_glob{


  #
  #  Test getting files on the mass store using globbing
  #

  my ($testdir) = @_;         #root testing directory
  my ($testfile) = "testfile1.plain";
  my ($testfile2) = "testfile2.plain";

  #do the get using globbing
  `./archive get -C $testdir "*.plain"`;
  $CHILD_ERROR == 0 || 
    die ("archive get \"*.plain\" returned nonzero exit status");

  #check that get worked.
  -e $testfile   || die ("$testfile not gotten in archive get \"*.plain\"");
  -e $testfile2  || die ("$testfile2 not gotten in archive get \"*.plain\"");

  #clean up
  unlink ($testfile);
  unlink ($testfile2);

}





sub test_get_glob_to_file_and_dir{


  #
  # init_get_tests made a file called isafile and a directory called isadir
  # This test will do an archive get "isa*", which will resolve into
  # archive get isafile isadir
  # It should get the file, ignore the dir, and return with a nonzero exit
  # status.
  #

  my ($testdir) = @_;         #root testing directory
  my ($lsoutput);             #output of ls
  my (@lslines);              #used to parse $lsoutput
  my ($ugfile) = "isafile";   #the (unglobbed) file to bring back  
  my ($ugdir) = "isadir";     #the (unglobbed) dir to ignore.


  #do the get using globbing
  `./archive get -C $testdir "isa*" 2> /dev/null`;
  $CHILD_ERROR != 0 || 
    die ("archive get \"isa*\" returned zero exit status");

  #check that get worked.
  -e $ugfile   || die ("$ugfile not gotten in archive get \"isa*\"");
  !(-e $ugdir)  || die ("$ugdir gotten in archive get \"isa*\"");

  #clean up
  unlink ($ugfile);

}





sub test_get_no_such_path{

  #
  #  Assure that archive get does something nice if given a bogus path.
  #  It should return a nonzero exit status.
  #

  my ($testdir) = @_;                 #root testing directory
  my ($subdir) = "/idontexist";       #nonexistent remote subdirectory
  my ($testfile) = "filedoesntexist"; #nonexistent file

  #set remote subdirectory
  $testdir=$testdir.$subdir;

  #attempt the get (should fail)
  `./archive get -C $testdir $testfile 2>/dev/null`;
  $CHILD_ERROR != 0 ||
     die ("archive get -C with a nonexistent path returned a zero exit status");     

  #check get's effect
  !(-e $testfile) || 
   die ("archive get manufactured a file from a nonexistent remote path");
 
  #clean up
  unlink ($testfile);

}





sub test_get_no_such_file{

  #
  #  Assure that archive get does something nice if given a bogus file and
  #   a real path.  It should exit with a nonzero status.
  #

  my ($testdir) = @_;                 #root testing directory
  my ($testfile) = "filedoesntexist"; #nonexistent file

  #attempt the get
  `./archive get -C $testdir $testfile 2> /dev/null`;
  $CHILD_ERROR != 0 || 
   die ("archive get of a nonexistent file returned with a zero exit status");

  #check get's effect
  !(-e $testfile) || 
   die ("archive get of a nonexistent file somehow created a local file");

  #clean up
  unlink ($testfile);

}




sub test_get_tar{


  #
  #  Test getting a file and untarring it (-x)
  #

  my ($testdir) = @_;  #root testing directory
  my ($tarfile) = "tarfile.tar";       #tarfile to get
  #files that should be in the tarfile (see init_get_tests)
  my ($testfile1) = "testfile1.plain";     
  my ($testfile2) = "testfile2.plain";

  #do the get
  `./archive get -C $testdir -x $tarfile`;
  $CHILD_ERROR == 0 || die ("archive get -x returned a nonzero exit status");

  #check that we got thef files, and that the local tarfile is gone.
  -e $testfile1  || die ("$testfile1 not retrieved from $tarfile");
  -e $testfile2  || die ("$testfile2 not retrieved from $tarfile");
  !(-e $tarfile) || die ("$tarfile is left in the local space");

  #clean up
  unlink ($tarfile);
  unlink ($testfile1);
  unlink ($testfile2);

}




sub test_get_tar_S{


  #
  #  Test getting a file and untarring it, leaving the local tarfile
  #    -x -S
  #

  my ($testdir) = @_;  #root testing directory
  my ($tarfile) = "tarfile.tar";       #tarfile to get
  #files that should be in the tarfile (see init_get_tests)
  my ($testfile1) = "testfile1.plain";     
  my ($testfile2) = "testfile2.plain";

  #do the get
  `./archive get -C $testdir -x -S $tarfile`;
  $CHILD_ERROR == 0 || die ("archive get -x -S returns nonzero error status");

  #check that we got thef files, and that the local tarfile is gone.
  -e $testfile1  || die ("$testfile1 not retrieved from $tarfile");
  -e $testfile2  || die ("$testfile2 not retrieved from $tarfile");
  -e $tarfile    || die ("$tarfile is not left in the local space");

  #clean up
  unlink ($tarfile);
  unlink ($testfile1);
  unlink ($testfile2);

}






sub test_get_gz{


  #
  #  Test getting a file and unzipping it (-z)
  #

  my ($testdir) = @_;       #root testing directory
  my ($testfile1) = "testfile3.gz";   #the zipped file that we will get
  my ($testfile_unzipped) = "testfile3";  #the unzipped version of testfile1

  #do the get 
  `./archive get -C $testdir -z $testfile1`;
  $CHILD_ERROR == 0 || die ("archive get -z returned nonzero error status");

  #test the results of get
  -e $testfile_unzipped   || 
      die ("archive get -z failed: Unzipped file not found in local space.");
  !(-e $testfile1)  ||
      die ("archive get -z failed: Zipped file still in local space");

  #clean up
  unlink ($testfile_unzipped);

}




sub test_get_tgz{


  #
  #  Test getting a file and untarring and unzipping it (-x of a .tgz file)
  #

  my ($testdir) = @_;  #root testing directory
  my ($tarfile) = "gztarfile.tgz";       #tarfile to get unzip and untar
     #files in the .tgz file according to init_get_tests
  my ($testfile1) = "testfile1.plain";
  my ($testfile2) = "testfile2.plain";

  #do the get
  `./archive get -C $testdir -x $tarfile`;
  $CHILD_ERROR == 0 || 
     die ("archive get -x file.tgz returned nonzero exit status");

  #check that the proper files are in the local space.
  !(-e $tarfile)  || die ("archive get -x left tarfile in local space");
  -e $testfile1   || die ("archive get -x did not produce $testfile1");
  -e $testfile2   || die ("archive get -x did not produce $testfile2");

  #clean up
  unlink ($tarfile);
  unlink ($testfile1);
  unlink ($testfile2);

}





sub test_get_stargz{


  #
  #  Test archive get -x "*gz"
  #  It should get two files.  One .tgz and one .gz
  #  The .tgz should get unzipped & untarred & deleted, the .gz should stay
  #    the same.
  #

  my ($testdir) = @_;  #root testing directory
  my ($tarfile) = "gztarfile.tgz";       #remote .tgz file to get
  my ($zipfile) = "testfile3.gz";       #remote .gz file to get
     #files in the .tgz file according to init_get_tests
  my ($testfile1) = "testfile1.plain";
  my ($testfile2) = "testfile2.plain";

  #do the get
  `./archive get -C $testdir -x "*gz"`;
  $CHILD_ERROR == 0 || die("get -x \"*gz\" returned nonzero exit status");

  #check that the correct files are left int the local space
  !(-e $tarfile)  || die ("get -x \"*gz\" left the the .tgz file");
  -e $testfile1   || die ("get -x \"*gz\" failed to produce $testfile1");
  -e $testfile2   || die ("get -x \"*gz\" failed to produce $testfile2");
  -e $zipfile     || die ("get -x \"*gz\" failed to produce $zipfile");

  #clean up
  unlink ($testfile1);
  unlink ($testfile2);
  unlink ($zipfile);

}





sub test_get_and_unzip_stargz{


  #
  #  Test archive get -x -z "*gz"
  #  It should get two files.  One .tgz and one .gz
  #  The .tgz should get unzipped & untarred & deleted, the .gz should also
  #    get untarred.
  #

  my ($testdir) = @_;  #root testing directory
  my ($tarfile) = "gztarfile.tgz";       #name of the .tgz file to get
  my ($zipfile) = "testfile3.gz";       #name of the .gz file to get
  my ($testfile1) = "testfile1.plain";     #file in the .tgz
  my ($testfile2) = "testfile2.plain";     #file in the .tgz
  my ($unzipped) = "testfile3";      #the unzipped version of $zipfile

  #do the get
  `./archive get -C $testdir -x -z "*gz"`;
  $CHILD_ERROR == 0 || 
     die ("archive get -x -z \"*gz\" returned nonzero exit status");

  #test that the correct files were produced
  !(-e $tarfile) || die ("archive get -x -z \"*gz\" retained the .tgz locally");
  !(-e $zipfile) || die ("archive get -x -z \"*gz\" retained the .gz locally");
   -e $testfile1 || 
                 die ("archive get -x -z \"*gz\" failed to produce $testfile1");
   -e $testfile2 || 
                 die ("archive get -x -z \"*gz\" failed to produce $testfile2");
   -e $unzipped || 
                 die ("archive get -x -z \"*gz\" failed to produce $unzipped");

  #clean up
  unlink ($testfile1);
  unlink ($testfile2);
  unlink ($unzipped);

}




sub test_get_migrate{


  #
  #  Test getting a file on the mass store with the -M option (which should
  #   migrate the remote file once the get has been completed.
  #

  my ($testdir) = @_;  #root testing directory
  my ($testfile) = "testfile1.plain";      #file to get and migrate.
  my (@filenames);
  my ($user,$host,$home);
  my ($setofhosts,@hostlist);
  my ($migrated);

  #do the get/migrate
  `./archive get -C $testdir -M $testfile 2>/dev/null`;
  $CHILD_ERROR == 0 || die ("archive get -M returned nonzero exit status");


  #check local file:
  -e $testfile || die ("get -M failed to get the file $testfile");

  #check remote migration:
  # because we are using the local.pm routine site_get_file_mode directly,
  # we have to set a few variables that archive would normally set for us.
  $user = $ENV{'USER'};

  $setofhosts = site_get_archive_host($user);
  @hostlist=split(",",$setofhosts);
  $host = $hostlist[0];

  $home = site_get_archive_home($user);
  # was file migrated?
  $migrated = site_file_is_migrated($user,$host,"$home/$testdir/$testfile");
  $migrated || die ("gut -M failed to migrate the remote file");

  #clean up
  unlink ($testfile);

}


