Stdout / Stderr Munging

Unix applications use the standard output and standard error stream for presenting messages to the user and logging. In the batch-processing nature of the cluster environment, this output cannot be seen as it is written by the program but only after the execution. By default, the cluster management system creates files ${JOB_NAME}.o${JOB_ID} and ${JOB_NAME}.e${JOB_ID} for standard output and error streams.

Merging Stdout and Stderr

Merging stdout and stderr is useful to half the number of log files running around. One cannot determine any more which messages were written to stdout and which were written to stderr. However, since most jobs in a cluster environment will directly write large files to disk instead of writing the results to stderr, merging stdout and stderr is useful.

Merging can be done using the -j parameter. The optional argument to this parameter gives the "merge direction", e.g. -j oe to merge standard out and standard error to standard out.

$ qsub -j oe job.sh

#!/bin/bash
...
#PBS -j oe
...

Log File Names

You can assign explicit names to the standard output and standard error log files. This, however, means that the log files are overwritten when re-executing the job. It is much more useful to set the job name. Then, the system will generate files ${JOB_NAME}.o${JOB_ID} and ${JOB_NAME}.e${JOB_ID} for standard output and error streams.

$ qsub -N job_name job.sh

#!/bin/bash
...
#PBS -N job_name
...

This of course can be combined with standard out and error merging:

$ qsub -N job_name -j oe job.sh

#!/bin/bash
...
#PBS -N job_name
#PBS -j oe
...