Skip to content

LLcopy2tmp

The LLcopy2tmp command can be used to copy files to a temporary directory located on a compute node's local disk.  Accessing files located on the local disk of a compute node is much faster than accessing files located on the shared Lustre file system or the file system where users' home directories are located.

When your job is launched by the scheduler, the environment variable $TMPDIR is automatically set and exported.  The value of $TMPDIR for each process is unique and references a temporary directory located on the compute node's local disk. While your job is running, your code and scripts can reference the environment variable $TMPDIR when it needs to access files in the temporary directory. See the page How to Use Environment Variables for instructions on how to use it in your code.

When the job is complete, the temporary directories are automatically deleted.

Usage: LLcopy2tmp [-gCopyGroup] <source1> [source2 ...]

This command will copy <source> and its contents to the local directory of the compute node pointed to by the TMPDIR environment variable.

<source> can be either a directory or a file. If it is a directory, then it will copy the directory and all of the files in the directory, but it will not copy subsequent directories within the <source> directory.

This command will intelligently only copy once per node for the same CopyGroup, which will default to the environment variable $SLURM_JOB_ID.

This command will dereference symlinks.

The optional -g flag designates the group ownership that the directory and files will have once they are copied to $TMPDIR.

Examples

For each of the examples below, let's say we have directories and files in the following structure:

  • ~/mydata_dir
    • datafile1.dat
    • datafile2.dat
      • mydata_subdir1
      • datafile3.dat
      • datafile4.dat
    • mydata_subdir2
      • datafile5.dat
      • datafile6.dat

and TMPDIR=/state/partition1/user/userid1/

Example 1

LLcopy2tmp ~/mydata_dir/datafile1.dat

This will copy datafile1.dat to the local temporary directory at location /state/partition1/user/userid1/datafile1.dat/

Example 2

LLcopy2tmp ~/mydata_dir/

This will copy the mydata_dir directory, datafile1.dat and datafile2.dat files into /state/partition1/user/userid1/ so that it will contain:

  • /state/partition1/user/userid1/
    • mydata_dir
      • datafile1.dat
      • datafile2.dat

Example 3

LLcopy2tmp ~/mydata_dir/mydata_subdir1 ~/mydata_dir/mydata_subdir2

This will copy the mydata_subdir1 and mydata_subdir2 directories and their file contents, into /state/partition1/user/userid1/ so that it will contain:

  • /state/partition1/user/userid1
    • mydata_subdir1
      • datafile3.dat
      • datafile4.dat
    • mydata_subdir2
      • datafile5.dat
      • datafile6.dat