DokuWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
\\
**[[cluster:0|Back]]**





===== Job Arrays =====

Just have to document this. Very handy.

You can find detailed informaton at this **[[http://lsfdocs.wesleyan.edu/lsf6.1_complete_doc_set_html/lsf6.1_admin/G_jobarrays.html#32777|Link]]**

A job array makes it easy to manage a sequence of jobs with "shorthand" syntax.  It could be managing 20 jobs or 2,000 jobs, managed by a single command.  Once submitted the array job itself can be managed or the individual jobs that make up the job array.

MAX_JOB_ARRAY_SIZE=65534

can be set in ''lsb.params''.  The default is 1,000.


===== ''bsub -J'' =====

For example:

**''bsub -q idle -J "test[1-20]" -i ~/arrays/in.%I -o ~/arrays/out.%J.%I ~/array/my_array_job.sh''**

This single command will submit 20 jobs at once to queue idle.  The **%I** iterates form 1 to 20.  The **%J** assumes the JOBPID of the job array.  If submitted, you would see something like:

<code>
bjobs 1766
JOBID  USER   STAT  QUEUE  FROM_HOST    EXEC_HOST     JOB_NAME   SUBMIT_TIME
1766   hmeij  RUN   idle   swallowtail  compute-1-1   test[1]    Sep 09 12:34
1766   hmeij  RUN   idle   swallowtail  compute-1-17  test[2]    Sep 09 12:34
1766   hmeij  RUN   idle   swallowtail  compute-2-31  test[3]    Sep 09 12:34
1766   hmeij  RUN   idle   swallowtail  compute-1-10  test[4]    Sep 09 12:34
...
1766   hmeij  PEND  idle   swallowtail       -        test[18]   Sep 09 12:34
1766   hmeij  PEND  idle   swallowtail       -        test[19]   Sep 09 12:34
1766   hmeij  PEND  idle   swallowtail       -        test[20]   Sep 09 12:34
</code>

Nifty.

Your input files are dynamically matched ... so "test[1]" uses ~/arrays/in.1 and the output file is saved as ~/arrays/out.1766.1 etc.

You now can manage your job arrays: for example to abort the entire job array ''bkill 1766'' or address individual jobs within the array ''bkill 1766[18]''.



===== Samples =====

If it was working you could use these sample files.

  * create input files: ~/arrays/in.N with content (where N = 1 .. 20)

<code>
'N'
</code>

  * create ~/arrays/my_array_job.sh

<code>
#!/bin/bash
read i
echo $(($i*$i))
</code>

Submit.

\\
**[[cluster:0|Back]]**
DokuWiki

User Tools

Site Tools

Page Tools
