Add files via upload
Very new to coding and this is my first "big" project in modifying a script. So welcome to
This PBS version is modified mainly based on the LSF version, but also took other versions as reference. Main changed to create this version is build the job dependencies to guarantee the sequential steps of each job in the original juicer.sh script. #PBS -W depend=afterok:jobID headerline is used.
The launch stats steps and post processing steps are moved into two separate scripts to avoid the variable value problems (expansion) in multiple level of heredocs. These two scripts will be called from the main juicer.sh at the appropriate step.
As a summary, the follwing scripts have been modified:
juicer.sh
split_rmdups.awk
mega.sh
The following two scripts has been added (extracted from the juicer.sh)
launch_stats.sh
postprocessing.sh
All steps has been tested till the ARROWHEAD step.
*HiCCUP step has not been tested in this version because of some un-resolved error in loading Jcuda native libraries.
One important notice for successful run of this PBS version:
***The $groupname variable has to be maximally 7 characters long*****
This length may change according to your cluster setup. See below for the reason.
The jobs' dependency of this PBS version juicer is based on jobID of each step. And the jobID of each job is obtained through the qstat output based on the specific job Name, jobID_stepx= $(qstat |grep <specific job name string> |cut -c 1-7). Unfortunately, PBS can not use job Name in building job dependency. The job Name column in qstat output has a maximum length of 16 characters by default. When the job Name exceeds this max length, the beginning part of the job Name string will be replaced by “…” and then followed by the last 13 characters (see examples below). If the job name is not fully displayed, the jobID value may be null due to the failure at grep step. Each job Name contains the $groupname, which gives each run specific timestamp and avoid's disruption from other runs of juicer from the same user. A potential cause of this problem is the ${groupname} value being too long. I have added comments in the juicer.sh script for setting this variable.
Below is a example of job names being too long:
$qstat
Job ID Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
2294690.pbs STDIN mzhibo 00:00:15 R batch
2294778.pbs make4Cbedgraph mzhibo 0 R batch
2294784.pbs ...dgraph1111111 mzhibo 0 Q batch
$
Loading
Please sign in to comment