Progression of Analysis in GCTA

elias
New Member

Posts: 9

Progression of Analysis in GCTA Feb 7, 2016 18:45:41 GMT

Quote

Post by elias on Feb 7, 2016 18:45:41 GMT

Hi!

Good morning!

I am currently running a GREML-LDMS analysis on 27K samples and 13M SNPs.

This is the command line I submitted:

gcta64 --bfile new_GCTA-GWAS.1000g_20M-imputation.QC --ld-score-region 200 --out new_GCTA-GWAS.1000g_20M-imputation.QCb

and here is the current progression of the analysis:

*******************************************************************
* Genome-wide Complex Trait Analysis (GCTA)
* version 1.25.2
* (C) 2010-2013 Jian Yang, Hong Lee, Michael Goddard and Peter Visscher
* The University of Queensland
* MIT License
*******************************************************************
Analysis started: Mon Feb 1 10:40:29 2016

Options:
--bfile new_GCTA-GWAS.1000g_20M-imputation.QC
--ld-score-region
--thread-num 2
--out new_GCTA-GWAS.1000g_20M-imputation.QC

Note: the program will be running on 2 threads.

Reading PLINK FAM file from [new_GCTA-GWAS.1000g_20M-imputation.QC.fam].
27065 individuals to be included from [new_GCTA-GWAS.1000g_20M-imputation.QC.fam].
Reading PLINK BIM file from [new_GCTA-GWAS.1000g_20M-imputation.QC.bim].
13502839 SNPs to be included from [new_GCTA-GWAS.1000g_20M-imputation.QC.bim].
Reading PLINK BED file from [new_GCTA-GWAS.1000g_20M-imputation.QC.bed] in SNP-major format ...
Genotype data for 27065 individuals and 13502839 SNPs to be included from [new_GCTA-GWAS.1000g_20M-imputation.QC.bed].

Calculating LD score between SNPs (block size of 10000Kb with an overlap of 5000Kb between blocks); LD rsq threshold = 0) ...
Calculating allele frequencies ...

I was wondering if it would be possible to know the progression of the analysis like PLINK does. I would like to know if the analysis is 10% or 80% complete.
This job has been running for over 5 days and our cluster can only handle up to 2 parallel nodes per job.

This is the memory currently use:

cpu=183:55:40, mem=153750581.57229 GB s, io=85.37757 GB, vmem=236.145G, maxvmem=275.522G

and my current hard resource:

hard resource_list: h_vmem=400G,h_rt=720000

It would be really useful to implement the percentage progression in the next version of GCTA.

Thank you for letting me know if there is anyway I can improve my job request.

Best,

Elias

Jian Yang Administrator Posts: 362	Progression of Analysis in GCTA Feb 9, 2016 6:27:35 GMT Quote Select Post Deselect Post Link to Post Member Give Gift Back to Top Post by Jian Yang on Feb 9, 2016 6:27:35 GMT It would be more efficient if you run the analysis on each chromosome separately and use more threads for each job.

elias
New Member

Posts: 9

Progression of Analysis in GCTA Feb 14, 2016 17:44:37 GMT via mobile

Quote

Post by elias on Feb 14, 2016 17:44:37 GMT

Hi Jian!

Thanks for your prompt reply.

I took you recommendation and run this analysis on 22 chromosomes using 4 thread and still get no information regarding the progress of the analysis. The analysis seems to take a while calculating allele frequency per chromosome as well. I have been running this analysis over a week now and the progress is not apparent.

Do you have an another suggestion when running this number of samples and SNPs?

Thank you!

elias
New Member

Posts: 9

Progression of Analysis in GCTA Feb 17, 2016 18:45:59 GMT

Quote

Post by elias on Feb 17, 2016 18:45:59 GMT

Hi Jian!

I am wondering what kind of parameter did you use to run the GREML-LDMS analysis in your recent paper (PMID 26323059).

When I increase the number of thread to 8 I got a segmentation faut.

This analysis either requires an extreme amount of computational memory or something is wrong in my command.

I run the past few days chromosome 22 only (250K SNPs; N=27K) with 2/4/8 thread with 200G/400G/800G of memory each time and all my jobs stop after Calculating Allele Frequency got a segmentation fault.

Thanks for your help!

Elias

Last Edit: Feb 17, 2016 18:46:36 GMT by elias

elias
New Member

Posts: 9

Progression of Analysis in GCTA Feb 18, 2016 6:56:24 GMT

Quote

Post by elias on Feb 18, 2016 6:56:24 GMT

Here is a job I submitted this morning and the segmentation fault I encountered:

h_vmem=100G
h_rt=100:00:00

gcta64 --bfile chr21.QC.GCTA-GWAS.1000G-Imp --ld-score-region 200 --out chr21.QC.GCTA-GWAS.1000G-Imp_0

*******************************************************************
* Genome-wide Complex Trait Analysis (GCTA)
* version 1.25.2
* (C) 2010-2013 Jian Yang, Hong Lee, Michael Goddard and Peter Visscher
* The University of Queensland
* MIT License
*******************************************************************
Analysis started: Wed Feb 17 14:01:15 2016

Options:
--bfile chr21.QC.GCTA-GWAS.1000G-Imp
--ld-score-region
--out chr21.QC.GCTA-GWAS.1000G-Imp_0

Note: This is a multi-thread program. You could specify the number of threads by the --thread-num option to speed up the computation if there are mu
ltiple processors in your machine.

Reading PLINK FAM file from [chr21.QC.GCTA-GWAS.1000G-Imp.fam].
27065 individuals to be included from [chr21.QC.GCTA-GWAS.1000G-Imp.fam].
Reading PLINK BIM file from [chr21.QC.GCTA-GWAS.1000G-Imp.bim].
146818 SNPs to be included from [chr21.QC.GCTA-GWAS.1000G-Imp.bim].
Reading PLINK BED file from [chr21.QC.GCTA-GWAS.1000G-Imp.bed] in SNP-major format ...
Genotype data for 27065 individuals and 146818 SNPs to be included from [chr21.QC.GCTA-GWAS.1000G-Imp.bed].

Calculating LD score between SNPs (block size of 10000Kb with an overlap of 5000Kb between blocks); LD rsq threshold = 0) ...
Calculating allele frequencies ...
Calculating regional mean LD score (region width = 200Kb with an overlap of 100Kb between regions) ...

/srv/gsfs0/admin_stuff/spe/scg3-feb2015/spool/scg3-2-8/job_scripts/3621194: line 2: 121136 Segmentation fault gcta64 --bfile chr21.QC.GCTA-GWAS.1000G-Imp --ld-score-region 200 --out chr21.QC.GCTA-GWAS.1000G-Imp_0

Here is the job log:

exit_status 139 (Segmentation fault)
ru_wallclock 29537s
ru_utime 28155.362s
ru_stime 839.776s
ru_maxrss 26.748MB
ru_ixrss 0.000B
ru_ismrss 0.000B
ru_idrss 0.000B
ru_isrss 0.000B
ru_minflt 529890
ru_majflt 5
ru_nswap 0
ru_inblock 1966672
ru_oublock 208
ru_msgsnd 0
ru_msgrcv 0
ru_nsignals 0
ru_nvcsw 114
ru_nivcsw 2929188
cpu 28995.138s
mem 593.167KGBs
io 0.929GB
iow 0.000s
maxvmem 26.738GB
arid undefined
ar_sub_time undefined
category -u elyas -l h_rt=504000,h_vmem=100G

I run this job with 2/3/4/8 nodes with 100G to 800G and encountered Seg fault as well.

I would truly appreciate your suggestion.

Best,

Elias

Jian Yang
Administrator

Posts: 362

Progression of Analysis in GCTA Feb 18, 2016 7:17:32 GMT

Quote

Post by Jian Yang on Feb 18, 2016 7:17:32 GMT

It seems that there is a memory leak when calculating the mean LD score of each segment. How about running the analysis in two steps?

1) Calculating the LD score for each SNP
gcta64 --bfile chr21.QC.GCTA-GWAS.1000G-Imp --ld-score --out chr21.QC.GCTA-GWAS.1000G-Imp_0 --thread-num 20

2) Calculating the mean LD score for each segment using the output file from the analysis above as the input
gcta64 --ld-file chr21.QC.GCTA-GWAS.1000G-Imp_0.score.ld --ld-score-region --out chr21.QC.GCTA-GWAS.1000G-Imp_0.region
This step should be very false since it only read per-SNP LD score to calculate the mean LD score for each segment.

Last Edit: Feb 18, 2016 7:18:59 GMT by Jian Yang

elias
New Member

Posts: 9

Progression of Analysis in GCTA Feb 24, 2016 3:42:14 GMT

Quote

Post by elias on Feb 24, 2016 3:42:14 GMT

Hi Jian,

Thanks for your reply.

I run these two commands.
the first one run very well and got the LD score for each SNP after few hours (with --thread-num 10) but the second command had also a segmentation fault after few second when calculating the mean LD score for each segment:

*******************************************************************
* Genome-wide Complex Trait Analysis (GCTA)
* version 1.25.2
* (C) 2010-2013 Jian Yang, Hong Lee, Michael Goddard and Peter Visscher
* The University of Queensland
* MIT License
*******************************************************************
Analysis started: Tue Feb 23 17:19:16 2016

Options:
--ld-file chr21.QC.GCTA_GWAS.1000G_Imp_0.score.ld
--ld-score-region
--out chr21.QC.GCTA_GWAS.1000G_Imp_0.region

Note: This is a multi-thread program. You could specify the number of threads by the --thread-num option to speed up the computation if there are multiple processors in your machine.

Reading per-SNP LD score from [chr21.QC.GCTA_GWAS.1000G_Imp_0.score.ld] ...
Per-SNP LD score for 146818 SNPs read from [chr21.QC.GCTA_GWAS.1000G_Imp_0.score.ld].
Calculating regional mean LD score (region width = 200Kb with an overlap of 100Kb between regions) ...

srv/gsfs0/admin_stuff/sge/scg3-feb2015/spool/scg3-1-4/job_scripts/3642046: line 3: 60606 Segmentation fault gcta64 --ld-file chr21.QC.GCTA_GWAS.1000G_Imp_0.score.ld --ld-score-region --out chr21.QC.GCTA_GWAS.1000G_Imp_0.region

Is there another way to get the LD for each segment?

Jian Yang Administrator Posts: 362	Progression of Analysis in GCTA Feb 24, 2016 5:36:07 GMT Quote Select Post Deselect Post Link to Post Member Give Gift Back to Top Post by Jian Yang on Feb 24, 2016 5:36:07 GMT If you wish, you can upload the "chr21.QC.GCTA_GWAS.1000G_Imp_0.score.ld" file somewhere. I will have a look.

Post by elias on Feb 7, 2016 18:45:41 GMT

Post by Jian Yang on Feb 9, 2016 6:27:35 GMT

Post by elias on Feb 14, 2016 17:44:37 GMT

Post by elias on Feb 17, 2016 18:45:59 GMT

Post by elias on Feb 18, 2016 6:56:24 GMT

Post by Jian Yang on Feb 18, 2016 7:17:32 GMT

Post by elias on Feb 24, 2016 3:42:14 GMT

Post by Jian Yang on Feb 24, 2016 5:36:07 GMT