Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile test, singularity Error on HPC - could not extract squashfs data, unsquashfs not found #702

Closed
djbradshaw2 opened this issue Oct 23, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@djbradshaw2
Copy link

Description of the bug

Dear Nextflow Mag Pipeline developers,

Thank you for such a great tool! While trying to run the test mag dataset using singularity, the pipeline fails very quickly with an Exit Status 255 and an error regarding not being able to extract squashfs data due to unsquashfs not being found. This is despite modules of nextflow, singularity, and unsquashfs being loaded and unsquashfs command being recognized and functional when run on the command line. This is being run on an HPC. Please see the attached nextflow.log

Please let me know if you have any questions or need any other information. Thank you for your time and help.

Sincerely,

David

Command used and terminal output

module load nextflow
module load singularity
module load squashfs-4.4-gcc-9.2.0-7d3vq45

nextflow run nf-core/mag -r 3.1.0 -profile test,singularity --outdir test_output.out

Pulling Singularity image https://depot.galaxyproject.org/singularity/bowtie2:2.4.2--py38h1c8e9b9_1 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-bowtie2-2.4.2--py38h1c8e9b9_1.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/krona:2.7.1--pl526_5 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-krona-2.7.1--pl526_5.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/ubuntu:20.04 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-ubuntu-20.04.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/fastp:0.23.4--h5f740d0_0 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-fastp-0.23.4--h5f740d0_0.img]
Pulling Singularity image https://depot.galaxyproject.org/singularity/fastqc:0.12.1--hdfd78af_0 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-fastqc-0.12.1--hdfd78af_0.img]
Staging foreign file: https://busco-data.ezlab.org/v5/data/lineages/bacteria_odb10.2024-01-08.tar.gz
Execution cancelled -- Finishing pending tasks before exit
Pulling Singularity image https://depot.galaxyproject.org/singularity/pandas:1.4.3 [cache /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-pandas-1.4.3.img]
-[nf-core/mag] Pipeline completed with errors-
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /mnt/beegfs/home/dbradshaw2015/work/singularity -- Use the environment variable NXF_SINGULARITY_CACHEDIR to specify a different location
ERROR ~ Error executing process > 'NFCORE_MAG:MAG:BUSCO_QC:BUSCO_DB_PREPARATION (bacteria_odb10.2024-01-08.tar)'

Caused by:
Process NFCORE_MAG:MAG:BUSCO_QC:BUSCO_DB_PREPARATION (bacteria_odb10.2024-01-08.tar) terminated with an error exit status (255)

Command executed:

mkdir buscodb
tar -xf bacteria_odb10.2024-01-08.tar.gz -C buscodb

cat <<-END_VERSIONS > versions.yml
"NFCORE_MAG:MAG:BUSCO_QC:BUSCO_DB_PREPARATION":
tar: $(tar --version 2>&1 | sed -n 1p | sed 's/tar (GNU tar) //')
END_VERSIONS

Command exit status:
255

Command output:
(empty)

Command error:
INFO: Convert SIF file to sandbox...
FATAL: while extracting /mnt/beegfs/home/dbradshaw2015/work/singularity/depot.galaxyproject.org-singularity-ubuntu-20.04.img: root filesystem extraction failed: could not extract squashfs data, unsquashfs not found

Work dir:
/mnt/beegfs/home/dbradshaw2015/work/ef/1e6db689ccf7adc9357eaaee2efef5

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

-- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

-- Check '.nextflow.log' file for details

Relevant files

No response

System information

Nextflow version: 24.04.4
Hardware: HPC
Executor: local
Container: Singularity (3.4.1)
OS: Linux
nf-core/mag: 3.1.0

@djbradshaw2 djbradshaw2 added the bug Something isn't working label Oct 23, 2024
@jfy133
Copy link
Member

jfy133 commented Oct 24, 2024

Hi @djbradshaw2

Unfortunately this appears to be an issue with your singularity installation or configuration, as it comes up a lot with singularity itself rather than Nextflow nor the pipeline, e.g. apptainer/singularity#5711.

You might need to check the installation again that it's working.
But it also looks like you're running a very old version of Singularity which may also be causing an issue - you should update/request an update to this, and/or installation of the more modern and maintained 'Apptainer' fork of SIngularity

@jfy133 jfy133 closed this as completed Oct 24, 2024
@djbradshaw2
Copy link
Author

Hi @jfy133,

Thank you very much for the information and guidance! Will request an update of singularity/installation of apptainer, pending my OIT's preference.

Sincerely,

David

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants