Skip to content

Finding pMatlab Output

When you run a pMatlab job, the standard output of each process is captured in out files. The .out filenames include a number indicating the process id that created the file: <pMatlab_script_name>.<pid>.out where:

  • <Matlab_script_name> is the filename of your pMatlab script
  • <pid> is the id of the process

Depending on how you submit your job the location of the files will be in either of these 2 locations:

  • The ./MatMPI directory (which is created in your working directory)
  • In subdirectories within the MatMPI directory:
    • ./MatMPI/p<start-pid>-p<end-pid>_<compute node name> where:
      • p<start-pid> is the id of the first process running on the compute node whose name is <compute node name>
      • p<end-pid> is the id of the last process running on the compute node whose name is <compute node name>
    • If your job ran on multiple compute nodes, the log files will be spread across multiple subdirectories, each one containing the compute node name in the subdirectory's name.

The name of the output files will be <pMatlab_script_name>.<pid>.out.

Where will the .out files reside?

Default location

In most cases, the .out files reside in the subdirectories that include the process id and compute node name within the MatMPI directory:

./MatMPI/p<start-pid>-p<end-pid>_<compute node name>/<pMatlab_script_name>.<pid>.out

For example, this is the path and file name for a job named param_sweep_parallel_v2 that ran on 4 processes on compute node b-11-16-3:

./MatMPI/p0-p3_b-11-16-3/param_sweep_parallel_v2.1.out

If you set GRIDMATLAB_MANYCORE to 'NO'

If you set the GRIDMATLAB_MANYCORE environment variable to 'no' (the default setting is 'yes'), your output files will be located in the MatMPI directory:

./MatMPI/<pMatlab_script_name>.<pid>.out

For example:

./MatMPI/param_sweep_parallel_v2.1.out

The other files in ./MatMPI

pRUN_Parallel_Wrapper.err

The file pRUN_Parallel_Wrapper.err will be created in the MatMPI directory. This file contains all standard error output from your pMatlab job. Each line in the file is prefixed with the Pid where the error occurred which you can use to identify the .out file and investigate further. You'll find the same error message in the associated .out file.

Unix_Commands*.sh

The Unix_Commands*.sh files are the shell scripts that were used to launch your pMatlab job. This may be useful to the SuperCloud team to help diagnose any problems you may run into when you launch your job or during runtime.

If you set the GRIDMATLAB_MANYCORE environment variable to 'no' (the default setting is 'yes'), gridMatlab generates one Unix_Commands..out file for each process in your job. This could result in thousands of files and cause the Lustre filesystem to perform poorly. To try to prevent this, these files are automatically deleted after job completion unless you set the GRIDMATLAB_KEEP_TEMP_FILES environment variable as shown below:

setenv('GRIDMATLAB_KEEP_TEMP_FILES','ON');