I/O intensive jobs on a shared home directory
- Details Hits: 4948
Our default setup includes shared home directories exported via NFS. If you have I/O intensive apps you may have a bottleneck. (and yes, if you kill the storage system, your jobs get deleted). The trick is to know witch node is writing and to copy the input data locally.
Sun Grid Engine creates for each job, by default, a temporary directory. In our case this is a local directory /scratch/tmp/... . When you run a job this happens:
[heri@fep-53-2 ~]$ qsub -q ibm-quad.q
echo $TMPDIR
cp BigFile.blob $TMPDIR/
ls -l $TMPDIR
echo `hostname`
Your job 94740 ("STDIN") has been submitted
[heri@fep-53-2 ~]$ cat STDIN.o94740
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
/scratch/tmp/94740.1.ibm-quad.q
total 1572
-rw-rw-r-- 1 heri heri 1602381 Dec 7 01:42 BigFile.blob
quad-wn19.grid.pub.ro
But Carefull! If you run an MPI job, your script only runs on the master node 0.
If you really want to read in paralell from the local filesystem you could:
- either copy the files via NFS to each of my nodes (on second thought ... )
- copy via SSH to each of my nodes
For example, let's run an MPI job with the restriction of only one mpi process per node. (Full article here)
Prereq: passphraseless default key. (Full article here)