Hi Alfonso
thank you very much for following up, and sorry for the late reply!
This was some weeks back, so I hope I remember correctly. Also, my understanding of SGE is rather pedestrian (I'm a medical doctor, so computational stuff tends to befuddle me).
Essentially, there was a collision between an SGE job script I wrote and my /.cshrc. The script was a wrapper around a MATLAB script that ran SPM12 / CAT12 ahead of a CONN batch (for some bespoke preprocessing). The script and the .cshrc both loaded MATLAB, but different versions, and that let the whole thing stall.
Another problem I had is that this was a very large connectome (n=1000, Genome Superstruct Project data), and even if it ran, the .dmat files took ages to load. So I split the whole thing up, processed each subject in its own CONN project, and later merged with conn_merge.m
So, here is how the final SGE job script looked like (comments for this email underlined)
[u]Basic settings:[/u]
#!/bin/tcsh#$ -N matlab_job
#$ -o /data/group/epilepsy/Abela/logs/
#$ -e /data/group/epilepsy/Abela/logs/
#$ -q global
#$ -l h_vmem=20G
#$ -l h_rt=20:00:00
[u]Key step: purge any modules loaded by .cshrc[/u]
setenv SGE_CLEAN 1
# First unload any modules loaded by ~/.cshrc, then load the defaults
module purge
module load nan/default
module load sge
# Load script dependent modules here
module load matlab/9.5.0
module load spm/12-7487
[u]Actual wrapper: this takes a text file with patient indexes (file paths) and executes the matlab script netloc_conn for each of them[/u]
# set the working variables
set working_code=/data/group/epilepsy/Abela/code/conn
set sge_index=${working_code}/gsp_conn_index.txt
set sge_id=${working_code}/gsp_conn_subID.txt
# Don't included the .m extention
set matlabscript=netloc_conn_sub
# Extend the Matlab search to include your own .m files
if ( $?MATLABPATH ) then
setenv MATLABPATH ${MATLABPATH}:${working_code}
else
setenv MATLABPATH ${working_code}
endif
# Search the file for the SGE_TASK_ID number as a line number
setenv subj_dir 'sed -n ${SGE_TASK_ID}p ${sge_index}|awk '{print $1}''
setenv subj_id 'sed -n ${SGE_TASK_ID}p ${sge_id}|awk '{print $1}''
# Print out files and the Matlab running config
# Good for debugging
echo "-----------------------------------------------"
echo "Working file information"
echo "Selected subject :"${subj_id}
echo "Selected script :"${matlabscript}
echo "-----------------------------------------------"
#echo "Matlab running defaults"
#The defaults used before running your script below.
#matlab -n
Hope that makes sense! If you know of any trick to speed up the .dmat loading, I would be all ears!
Kind regards,
Eugenio
thank you very much for following up, and sorry for the late reply!
This was some weeks back, so I hope I remember correctly. Also, my understanding of SGE is rather pedestrian (I'm a medical doctor, so computational stuff tends to befuddle me).
Essentially, there was a collision between an SGE job script I wrote and my /.cshrc. The script was a wrapper around a MATLAB script that ran SPM12 / CAT12 ahead of a CONN batch (for some bespoke preprocessing). The script and the .cshrc both loaded MATLAB, but different versions, and that let the whole thing stall.
Another problem I had is that this was a very large connectome (n=1000, Genome Superstruct Project data), and even if it ran, the .dmat files took ages to load. So I split the whole thing up, processed each subject in its own CONN project, and later merged with conn_merge.m
So, here is how the final SGE job script looked like (comments for this email underlined)
[u]Basic settings:[/u]
#!/bin/tcsh#$ -N matlab_job
#$ -o /data/group/epilepsy/Abela/logs/
#$ -e /data/group/epilepsy/Abela/logs/
#$ -q global
#$ -l h_vmem=20G
#$ -l h_rt=20:00:00
[u]Key step: purge any modules loaded by .cshrc[/u]
setenv SGE_CLEAN 1
# First unload any modules loaded by ~/.cshrc, then load the defaults
module purge
module load nan/default
module load sge
# Load script dependent modules here
module load matlab/9.5.0
module load spm/12-7487
[u]Actual wrapper: this takes a text file with patient indexes (file paths) and executes the matlab script netloc_conn for each of them[/u]
# set the working variables
set working_code=/data/group/epilepsy/Abela/code/conn
set sge_index=${working_code}/gsp_conn_index.txt
set sge_id=${working_code}/gsp_conn_subID.txt
# Don't included the .m extention
set matlabscript=netloc_conn_sub
# Extend the Matlab search to include your own .m files
if ( $?MATLABPATH ) then
setenv MATLABPATH ${MATLABPATH}:${working_code}
else
setenv MATLABPATH ${working_code}
endif
# Search the file for the SGE_TASK_ID number as a line number
setenv subj_dir 'sed -n ${SGE_TASK_ID}p ${sge_index}|awk '{print $1}''
setenv subj_id 'sed -n ${SGE_TASK_ID}p ${sge_id}|awk '{print $1}''
# Print out files and the Matlab running config
# Good for debugging
echo "-----------------------------------------------"
echo "Working file information"
echo "Selected subject :"${subj_id}
echo "Selected script :"${matlabscript}
echo "-----------------------------------------------"
#echo "Matlab running defaults"
#The defaults used before running your script below.
#matlab -n
Hope that makes sense! If you know of any trick to speed up the .dmat loading, I would be all ears!
Kind regards,
Eugenio