Commit 1bc0ebce authored by Swati Parekh's avatar Swati Parekh
Browse files

Downsampling is modified and ExampleData is re-run

parent 067f4391

ExampleData/.DS_Store

0 → 100644
+6 KiB

File added.

No diff preview for this file type.

View file

ExampleData/._.DS_Store

0 → 100644
+4 KiB

File added.

No diff preview for this file type.

View file

ExampleData/example.Log.final.out

100644 → 100755
+4 −4
Original line number Diff line number Diff line 
                                 Started job on |	Jun 22 19:13:45

                             Started mapping on |	Jun 22 19:19:51

                                    Finished on |	Jun 22 19:20:09

       Mapping speed, Million of reads per hour |	170.66

                                 Started job on |	Jun 23 13:37:27

                             Started mapping on |	Jun 23 13:52:03

                                    Finished on |	Jun 23 13:52:55

       Mapping speed, Million of reads per hour |	59.07



                          Number of input reads |	853296

                      Average input read length |	50

ExampleData/example.Log.out

100644 → 100755
+51 −135
Original line number Diff line number Diff line
@@ -147,41 +147,41 @@ quantTranscriptomeBan IndelSoftclipSingleend 
twopass1readsN                    18446744073709551615

twopassMode                       None

##### Command Line:

STAR --genomeDir /data/share/ngs/genomes/Human/hg38/STAR5idx_noGTF/ --runThreadN 15 --readFilesCommand zcat --sjdbGTFfile /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf --outFileNamePrefix /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example. --outSAMtype BAM Unsorted --outSAMmultNmax 1 --outFilterMultimapNmax 50 --outSAMunmapped Within --sjdbOverhang 49 --twopassMode Basic --readFilesIn /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

STAR --genomeDir /data/ngs/genomes/Human/hg38/STAR5idx_noGTF/ --runThreadN 1 --readFilesCommand zcat --sjdbGTFfile /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf --outFileNamePrefix /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood. --outSAMtype BAM Unsorted --outSAMmultNmax 1 --outFilterMultimapNmax 50 --outSAMunmapped Within --sjdbOverhang 49 --twopassMode Basic --readFilesIn /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz

##### Initial USER parameters from Command Line:

outFileNamePrefix                 /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.

outFileNamePrefix                 /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.

###### All USER parameters from Command Line:

genomeDir                     /data/share/ngs/genomes/Human/hg38/STAR5idx_noGTF/     ~RE-DEFINED

runThreadN                    15     ~RE-DEFINED

genomeDir                     /data/ngs/genomes/Human/hg38/STAR5idx_noGTF/     ~RE-DEFINED

runThreadN                    1     ~RE-DEFINED

readFilesCommand              zcat        ~RE-DEFINED

sjdbGTFfile                   /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf     ~RE-DEFINED

outFileNamePrefix             /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.     ~RE-DEFINED

sjdbGTFfile                   /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf     ~RE-DEFINED

outFileNamePrefix             /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.     ~RE-DEFINED

outSAMtype                    BAM   Unsorted        ~RE-DEFINED

outSAMmultNmax                1     ~RE-DEFINED

outFilterMultimapNmax         50     ~RE-DEFINED

outSAMunmapped                Within        ~RE-DEFINED

sjdbOverhang                  49     ~RE-DEFINED

twopassMode                   Basic     ~RE-DEFINED

readFilesIn                   /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz        ~RE-DEFINED

readFilesIn                   /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz        ~RE-DEFINED

##### Finished reading parameters from all sources



##### Final user re-defined parameters-----------------:

runThreadN                        15

genomeDir                         /data/share/ngs/genomes/Human/hg38/STAR5idx_noGTF/

readFilesIn                       /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz   

runThreadN                        1

genomeDir                         /data/ngs/genomes/Human/hg38/STAR5idx_noGTF/

readFilesIn                       /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz   

readFilesCommand                  zcat   

outFileNamePrefix                 /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.

outFileNamePrefix                 /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.

outSAMtype                        BAM   Unsorted   

outSAMunmapped                    Within   

outSAMmultNmax                    1

outFilterMultimapNmax             50

sjdbGTFfile                       /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf

sjdbGTFfile                       /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf

sjdbOverhang                      49

twopassMode                       Basic



-------------------------------

##### Final effective command line:

STAR   --runThreadN 15   --genomeDir /data/share/ngs/genomes/Human/hg38/STAR5idx_noGTF/   --readFilesIn /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz      --readFilesCommand zcat      --outFileNamePrefix /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.   --outSAMtype BAM   Unsorted      --outSAMunmapped Within      --outSAMmultNmax 1   --outFilterMultimapNmax 50   --sjdbGTFfile /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf   --sjdbOverhang 49   --twopassMode Basic

STAR   --runThreadN 1   --genomeDir /data/ngs/genomes/Human/hg38/STAR5idx_noGTF/   --readFilesIn /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz      --readFilesCommand zcat      --outFileNamePrefix /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.   --outSAMtype BAM   Unsorted      --outSAMunmapped Within      --outSAMmultNmax 1   --outFilterMultimapNmax 50   --sjdbGTFfile /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf   --sjdbOverhang 49   --twopassMode Basic



##### Final parameters after user input--------------------------------:

versionSTAR                       20201
@@ -189,10 +189,10 @@ versionGenome 20101 20200 
parametersFiles                   -   

sysShell                          -

runMode                           alignReads

runThreadN                        15

runThreadN                        1

runDirPerm                        User_RWX

runRNGseed                        777

genomeDir                         /data/share/ngs/genomes/Human/hg38/STAR5idx_noGTF/

genomeDir                         /data/ngs/genomes/Human/hg38/STAR5idx_noGTF/

genomeLoad                        NoSharedMemory

genomeFastaFiles                  -   

genomeChainFiles                  -
@@ -201,7 +201,7 @@ genomeChrBinNbits 18 
genomeSAsparseD                   1

genomeSuffixLengthMax             18446744073709551615

genomeFileSizes                   0   

readFilesIn                       /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz   

readFilesIn                       /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz   

readFilesCommand                  zcat   

readMatesLengthsIn                NotEqual

readMapNumber                     18446744073709551615
@@ -216,7 +216,7 @@ limitOutSJcollapsed 1000000 
limitOutSJoneRead                 1000

limitBAMsortRAM                   0

limitSjdbInsertNsj                1000000

outFileNamePrefix                 /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.

outFileNamePrefix                 /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.

outTmpDir                         -

outTmpKeep                        None

outStd                            Log
@@ -317,7 +317,7 @@ chimOutType SeparateSAMold 
chimFilter                        banGenomicN   

chimSegmentReadGapMax             0

sjdbFileChrStartEnd               -   

sjdbGTFfile                       /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf

sjdbGTFfile                       /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf

sjdbGTFchrPrefix                  -

sjdbGTFfeatureExon                exon

sjdbGTFtagExonParentTranscript    transcript_id
@@ -333,13 +333,13 @@ twopassMode Basic 
----------------------------------------





   Input read files for mate 1, from input string /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

-rw-r--r-- 1 parekh common 40267824 Jun 22 19:13 /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

   Input read files for mate 1, from input string /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz

-rw-r--r-- 1 ziegenhain ngs 40267826 Jun 23 13:37 /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz



   readsCommandsFile:

exec > "/data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example._STARtmp/tmp.fifo.read1"

exec > "/data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood._STARtmp/tmp.fifo.read1"

echo FILE 0

zcat      "/data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz"

zcat      "/data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz"



Finished loading and checking parameters

Reading genome generation parameters:
@@ -813,7 +813,7 @@ Number of real (reference) chromosomes= 455 
453	chrX	156040895	3066036224

454	chrY	57227415	3222274048

455	chrY_KI270740v1_random	37240	3279683584

Started loading the genome: Thu Jun 22 19:13:45 2017

Started loading the genome: Fri Jun 23 13:37:27 2017



Checking Genome sizefile size: 3279945728 bytes; state: good=1 eof=0 fail=0 bad=0

Checking SA sizefile size: 25156855213 bytes; state: good=1 eof=0 fail=0 bad=0
@@ -827,136 +827,52 @@ Loading Genome ... done! state: good=1 eof=0 fail=0 bad=0; loaded 3279945728 byt 
SA file size: 25156855213 bytes; state: good=1 eof=0 fail=0 bad=0

Loading SA ... done! state: good=1 eof=0 fail=0 bad=0; loaded 25156855213 bytes

Loading SAindex ... done: 1565873619 bytes

Finished loading the genome: Thu Jun 22 19:14:10 2017

Finished loading the genome: Fri Jun 23 13:37:54 2017



alignIntronMax=alignMatesGapMax=0, the max intron size will be approximately determined by (2^winBinNbits)*winAnchorDistNbins=589824

Jun 22 19:14:10 ..... processing annotations GTF

Processing sjdbGTFfile=/data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf, found:

Jun 23 13:37:54 ..... processing annotations GTF

Processing sjdbGTFfile=/data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf, found:

		199184 transcripts

		1176808 exons (non-collapsed)

		348697 collapsed junctions

Jun 22 19:14:34 ..... finished GTF processing

Jun 22 19:14:34   Loaded database junctions from the GTF file: /data/share/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf: 348697 total junctions

Jun 23 13:38:19 ..... finished GTF processing

Jun 23 13:38:19   Loaded database junctions from the GTF file: /data/ngs/genomes/Human/hg38/Homo_sapiens.GRCh38.84_chrsNamesUCSC.gtf: 348697 total junctions



WARNING: long repeat for junction # 115413 : chr15 88855522 88855578; left shift = 14; right shift = 255

WARNING: long repeat for junction # 157788 : chr18 46969610 47029116; left shift = 183; right shift = 255

Jun 22 19:14:35   Finished preparing junctions

Jun 22 19:14:35 ..... inserting junctions into the genome indices

Jun 22 19:15:08   Finished SA search: number of new junctions=348621, old junctions=0

Jun 22 19:15:34   Finished sorting SA indicesL nInd=68329716

Jun 22 19:17:14   Finished inserting junction indices

Jun 22 19:17:41   Finished SAi

Jun 22 19:17:42 ..... finished inserting junctions into genome

Created thread # 1

Created thread # 2

Created thread # 3

Created thread # 4

Created thread # 5

Jun 23 13:38:20   Finished preparing junctions

Jun 23 13:38:20 ..... inserting junctions into the genome indices

Jun 23 13:46:29   Finished SA search: number of new junctions=348621, old junctions=0

Jun 23 13:46:57   Finished sorting SA indicesL nInd=68329716

Jun 23 13:48:45   Finished inserting junction indices

Jun 23 13:49:13   Finished SAi

Jun 23 13:49:13 ..... finished inserting junctions into genome

Starting to map file # 0

mate 1:   /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

Created thread # 6

Created thread # 7

Created thread # 8

Created thread # 9

Created thread # 10

Created thread # 11

Created thread # 12

Created thread # 13

Created thread # 14

Thread #4 end of input stream, nextChar=-1

Completed: thread #6

Completed: thread #7

Completed: thread #8

Completed: thread #11

Completed: thread #9

Completed: thread #13

Completed: thread #10

Completed: thread #12

mate 1:   /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz

Thread #0 end of input stream, nextChar=-1

Completed: thread #0

Completed: thread #14

Completed: thread #4

Completed: thread #1

Joined thread # 1

Completed: thread #2

Joined thread # 2

Completed: thread #3

Joined thread # 3

Joined thread # 4

Completed: thread #5

Joined thread # 5

Joined thread # 6

Joined thread # 7

Joined thread # 8

Joined thread # 9

Joined thread # 10

Joined thread # 11

Joined thread # 12

Joined thread # 13

Joined thread # 14

Jun 22 19:17:54   Loaded database junctions from the 1st pass file: /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example._STARpass1//SJ.out.tab: 368090 total junctions

Jun 23 13:49:58   Loaded database junctions from the 1st pass file: /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood._STARpass1//SJ.out.tab: 368090 total junctions



WARNING: long repeat for junction # 115413 : chr15 88855522 88855578; left shift = 14; right shift = 255

WARNING: long repeat for junction # 157788 : chr18 46969610 47029116; left shift = 183; right shift = 255

Jun 22 19:17:56   Finished preparing junctions

Jun 22 19:17:56 ..... inserting junctions into the genome indices

Jun 22 19:17:57   Finished SA search: number of new junctions=718, old junctions=348621

Jun 22 19:17:57   Finished sorting SA indicesL nInd=140694

Jun 22 19:19:42   Finished inserting junction indices

Jun 22 19:19:48   Finished SAi

Jun 22 19:19:48 ..... finished inserting junctions into genome

Jun 23 13:49:59   Finished preparing junctions

Jun 23 13:49:59 ..... inserting junctions into the genome indices

Jun 23 13:50:01   Finished SA search: number of new junctions=718, old junctions=348621

Jun 23 13:50:02   Finished sorting SA indicesL nInd=140694

Jun 23 13:51:56   Finished inserting junction indices

Jun 23 13:52:02   Finished SAi

Jun 23 13:52:02 ..... finished inserting junctions into genome



   Input read files for mate 1, from input string /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

-rw-r--r-- 1 parekh common 40267824 Jun 22 19:13 /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

   Input read files for mate 1, from input string /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz

-rw-r--r-- 1 ziegenhain ngs 40267826 Jun 23 13:37 /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz



   readsCommandsFile:

exec > "/data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example._STARtmp/tmp.fifo.read1"

exec > "/data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood._STARtmp/tmp.fifo.read1"

echo FILE 0

zcat      "/data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz"

zcat      "/data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz"



Created thread # 1

Created thread # 2

Created thread # 3

Created thread # 4

Starting to map file # 0

mate 1:   /data/share/common/scripts/UMI_pipeline/zUMIs/ExampleData/example.cdnaread.filtered.fastq.gz

Created thread # 5

Created thread # 6

Created thread # 7

Created thread # 8

Created thread # 9

Created thread # 10

Created thread # 11

Created thread # 12

Created thread # 13

Created thread # 14

Thread #5 end of input stream, nextChar=-1

Completed: thread #6

Completed: thread #7

Completed: thread #11

Completed: thread #12

Completed: thread #13

mate 1:   /data/share/htp/zUMIs_paper/newdownsampling/canicommit/isallgood.cdnaread.filtered.fastq.gz

Thread #0 end of input stream, nextChar=-1

Completed: thread #0

Completed: thread #14

Completed: thread #9

Completed: thread #10

Completed: thread #8

Completed: thread #5

Completed: thread #1

Joined thread # 1

Completed: thread #2

Joined thread # 2

Completed: thread #3

Joined thread # 3

Completed: thread #4

Joined thread # 4

Joined thread # 5

Joined thread # 6

Joined thread # 7

Joined thread # 8

Joined thread # 9

Joined thread # 10

Joined thread # 11

Joined thread # 12

Joined thread # 13

Joined thread # 14

ALL DONE!

ExampleData/example.Log.progress.out

100644 → 100755
+3 −3
Original line number Diff line number Diff line 
           Time    Speed        Read     Read   Mapped   Mapped   Mapped   Mapped Unmapped Unmapped Unmapped Unmapped

                    M/hr      number   length   unique   length   MMrate    multi   multi+       MM    short    other

Jun 22 19:17:42	Started 1st pass mapping

Jun 22 19:17:47     42.5       59045       50    62.4%     49.4     0.3%    32.4%     0.0%     0.0%     4.5%     0.6%

Jun 22 19:17:54	Finished 1st pass mapping

Jun 23 13:49:13	Started 1st pass mapping

Jun 23 13:49:23     71.7      199103       50    62.1%     49.3     0.3%    32.6%     0.0%     0.0%     4.5%     0.7%

Jun 23 13:49:58	Finished 1st pass mapping

ALL DONE!

CRA Git | Maintained and supported by SUSTech CRA and CCSE