Failed running the tutorial

[telatin]

Hello,
I tried following the tutorial. After creating the pipeline.yaml file and specifying that the samples were paired (only).
This is the step that failed:

Rscript /miniconda3/envs/ocms_16s/lib/python3.7/site-packages/ocms_16S-0.0.2-py3.7.egg/ocms16S/R/dada2_filter_and_trim.R                      
 --infile=/tmp/ubuntu/ctmpyhfpy38y/2DSS__1.fastq.1   \
 --paired \
 --maxN=0  --maxEE=2,2 --truncQ=2      --truncLen=250,160 \
 --trimLeft=0,0 --filtered-directory=filtered.dir

which resulted in:

invalid class “SRFilterResult” object: superclass "Mnumeric" not defined in the environment of the object's class

Probably some input sanitation (on reads length? on module versions as in issue?) would make it easier to debug the issues, thanks!

[nickilott]

Hi,
Thanks for raising this. I had a look through the thread that you linked to and may be an issue with R or package versions. Which version of R are you using and did you install using the conda env that is in the repo? Dada2 version needs updating in the conda environment .yaml so I will also look into doing this as newer versions aren't using SRFilter. I will create a branch for this update and see how it goes...

[telatin]

Thanks for the quick reply.
Yes, I used the env file from the repository, and as you say there might be conflicting libraries which could be pinned in the yaml file.
This is my conda env export:

name: ocms_16s
channels:
  - r
  - defaults
  - bioconda
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - _r-mutex=1.0.1=anacondar_1
  - aiohttp=3.8.3=py37h540881e_0
  - aiosignal=1.3.1=pyhd8ed1ab_0
  - apsw=3.33.0.r1=py37h851c80e_2
  - async-timeout=4.0.2=pyhd8ed1ab_0
  - asynctest=0.13.0=py_0
  - attrs=22.1.0=pyh71513ae_1
  - bcrypt=3.2.2=py37h540881e_0
  - binutils_impl_linux-64=2.36.1=h193b22a_2
  - binutils_linux-64=2.36=hf3e587d_33
  - bioconductor-biobase=2.46.0=r36h516909a_0
  - bioconductor-biocgenerics=0.32.0=r36_0
  - bioconductor-biocparallel=1.20.0=r36he1b5a44_0
  - bioconductor-biostrings=2.54.0=r36h516909a_0
  - bioconductor-dada2=1.14.0=r36he1b5a44_0
  - bioconductor-delayedarray=0.12.0=r36h516909a_0
  - bioconductor-genomeinfodb=1.22.0=r36_0
  - bioconductor-genomeinfodbdata=1.2.2=r36_0
  - bioconductor-genomicalignments=1.22.0=r36h516909a_0
  - bioconductor-genomicranges=1.38.0=r36h516909a_0
  - bioconductor-iranges=2.20.0=r36h516909a_0
  - bioconductor-rhtslib=1.18.0=r36hdb70ac9_1
  - bioconductor-rsamtools=2.2.0=r36he1b5a44_0
  - bioconductor-s4vectors=0.24.0=r36h516909a_0
  - bioconductor-shortread=1.44.0=r36he1b5a44_0
  - bioconductor-summarizedexperiment=1.16.0=r36_0
  - bioconductor-xvector=0.26.0=r36h516909a_0
  - bioconductor-zlibbioc=1.32.0=r36h516909a_0
  - blas=1.1=openblas
  - boto3=1.26.8=pyhd8ed1ab_0
  - botocore=1.29.8=pyhd8ed1ab_0
  - brotlipy=0.7.0=py37h540881e_1004
  - bwidget=1.9.14=ha770c72_1
  - bzip2=1.0.8=h7f98852_4
  - c-ares=1.18.1=h7f98852_0
  - ca-certificates=2022.10.11=h06a4308_0
  - cachetools=5.2.0=pyhd8ed1ab_0
  - cairo=1.14.12=h8948797_3
  - certifi=2022.9.24=pyhd8ed1ab_0
  - cffi=1.15.0=py37h7f8727e_0
  - cgatcore=0.6.14=pyhdfd78af_0
  - charset-normalizer=2.1.1=pyhd8ed1ab_0
  - coreutils=9.1=h166bdaf_0
  - cryptography=38.0.2=py37h38fbfac_1
  - curl=7.68.0=hf8cf82a_0
  - drmaa=0.7.9=py_1000
  - fontconfig=2.13.1=hef1e5e3_1
  - freetype=2.12.1=hca18f0e_0
  - fribidi=1.0.10=h516909a_0
  - frozenlist=1.3.1=py37h540881e_0
  - ftputil=5.0.4=pyhd8ed1ab_0
  - gcc_impl_linux-64=7.5.0=habd7529_20
  - gcc_linux-64=7.5.0=h47867f9_33
  - gettext=0.21.1=h27087fc_0
  - gevent=22.10.1=py37haa10bde_0
  - gfortran_impl_linux-64=7.5.0=h56cb351_20
  - gfortran_linux-64=7.5.0=h78c8a43_33
  - glib=2.56.2=had28632_1001
  - google-api-core=2.10.2=pyhd8ed1ab_0
  - google-auth=2.14.1=pyh1a96a4e_0
  - google-cloud-core=2.3.2=pyhd8ed1ab_0
  - google-cloud-sdk=406.0.0=py37h89c1867_0
  - google-cloud-storage=2.6.0=pyh1a96a4e_0
  - google-crc32c=1.5.0=py37h5eee18b_0
  - google-resumable-media=2.4.0=pyhd8ed1ab_0
  - googleapis-common-protos=1.56.4=py37h89c1867_0
  - graphite2=1.3.14=h295c915_1
  - greenlet=1.1.3=py37hd23a5d3_0
  - grpc-cpp=1.48.1=h05bd8bd_1
  - grpcio=1.48.1=py37h3fae8ff_1
  - gsl=2.4=h294904e_1006
  - gxx_impl_linux-64=7.5.0=hd0bb8aa_20
  - gxx_linux-64=7.5.0=h555fc39_33
  - harfbuzz=1.9.0=he243708_1001
  - icu=58.2=hf484d3e_1000
  - idna=3.4=pyhd8ed1ab_0
  - importlib-metadata=4.11.4=py37h89c1867_0
  - jmespath=1.0.1=pyhd8ed1ab_0
  - jpeg=9e=h166bdaf_2
  - kernel-headers_linux-64=2.6.32=he073ed8_15
  - krb5=1.16.4=h2fd8d38_0
  - ld_impl_linux-64=2.36.1=hea4e1c9_2
  - lerc=4.0.0=h27087fc_0
  - libabseil=20220623.0=cxx17_h48a1fff_5
  - libblas=3.9.0=13_linux64_openblas
  - libcblas=3.9.0=13_linux64_openblas
  - libcrc32c=1.1.2=h9c3ff4c_0
  - libcurl=7.68.0=hda55be3_0
  - libdeflate=1.14=h166bdaf_0
  - libedit=3.1.20210910=h7f8727e_0
  - libev=4.33=h516909a_1
  - libffi=3.2.1=he1b5a44_1007
  - libgcc-devel_linux-64=7.5.0=hda03d7c_20
  - libgcc-ng=12.2.0=h65d4601_19
  - libgfortran-ng=7.5.0=h14aa051_20
  - libgfortran4=7.5.0=h14aa051_20
  - libgomp=12.2.0=h65d4601_19
  - libiconv=1.17=h166bdaf_0
  - liblapack=3.9.0=13_linux64_openblas
  - libopenblas=0.3.18=hf726d26_0
  - libpng=1.6.38=h753d276_0
  - libprotobuf=3.21.8=h6239696_0
  - libsodium=1.0.18=h516909a_1
  - libssh2=1.10.0=haa6b8db_3
  - libstdcxx-devel_linux-64=7.5.0=hb016644_20
  - libstdcxx-ng=12.2.0=h46fd767_19
  - libtiff=4.4.0=h55922b4_4
  - libuuid=1.41.5=h5eee18b_0
  - libuv=1.44.2=h166bdaf_0
  - libwebp-base=1.2.4=h166bdaf_0
  - libxcb=1.15=h7f8727e_0
  - libxml2=2.9.14=h74e7548_0
  - libzlib=1.2.13=h166bdaf_4
  - make=4.3=hd18ef5c_1
  - multidict=6.0.2=py37h540881e_1
  - ncurses=6.3=h27087fc_1
  - nomkl=3.0=0
  - numpy=1.21.6=py37h976b520_0
  - openblas=0.3.4=h9ac9557_1000
  - openssl=1.1.1s=h166bdaf_0
  - pandas=1.3.5=py37he8f5f7f_0
  - pandoc=2.19.2=h32600fe_1
  - pango=1.42.4=h049681c_0
  - paramiko=2.12.0=pyhd8ed1ab_0
  - pcre=8.45=h9c3ff4c_0
  - pip=22.3.1=pyhd8ed1ab_0
  - pixman=0.40.0=h36c2ea0_0
  - protobuf=4.21.8=py37hd23a5d3_0
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.21=pyhd8ed1ab_0
  - pynacl=1.5.0=py37h540881e_1
  - pyopenssl=22.1.0=pyhd8ed1ab_0
  - pysftp=0.2.9=py_1
  - pysocks=1.7.1=py37h89c1867_5
  - python=3.7.1=h381d211_1003
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - python_abi=3.7=2_cp37m
  - pytz=2022.6=pyhd8ed1ab_0
  - pyu2f=0.1.5=pyhd8ed1ab_0
  - pyyaml=6.0=py37h540881e_4
  - r-askpass=1.1=r36hcfec24a_2
  - r-assertthat=0.2.1=r36h6115d3f_0
  - r-backports=1.2.1=r36hcfec24a_0
  - r-base=3.6.1=h9bb98a2_1
  - r-base64enc=0.1_3=r36h96ca727_4
  - r-bh=1.75.0_0=r36hc72bb7e_0
  - r-bitops=1.0_7=r36hcfec24a_0
  - r-brio=1.1.2=r36hcfec24a_0
  - r-callr=3.7.0=r36hc72bb7e_0
  - r-catools=1.18.2=r36h03ef668_0
  - r-cli=2.5.0=r36hc72bb7e_0
  - r-colorspace=2.0_1=r36hcfec24a_0
  - r-crayon=1.4.1=r36hc72bb7e_0
  - r-crosstalk=1.1.1=r36hc72bb7e_0
  - r-curl=4.3.1=r36hcfec24a_0
  - r-data.table=1.14.0=r36hcfec24a_0
  - r-desc=1.3.0=r36hc72bb7e_0
  - r-diffobj=0.3.4=r36hcfec24a_0
  - r-digest=0.6.27=r36h03ef668_0
  - r-dplyr=1.0.2=r36h0357c0b_0
  - r-ellipsis=0.3.2=r36hcfec24a_0
  - r-evaluate=0.14=r36h6115d3f_2
  - r-fansi=0.4.2=r36hcfec24a_0
  - r-farver=2.1.0=r36h03ef668_0
  - r-formatr=1.9=r36hc72bb7e_0
  - r-futile.logger=1.4.3=r36h6115d3f_1003
  - r-futile.options=1.0.1=r36h6115d3f_0
  - r-gdata=2.18.0=r36h6115d3f_0
  - r-generics=0.1.0=r36hc72bb7e_0
  - r-getopt=1.20.3=r36h6115d3f_0
  - r-ggplot2=3.3.2=r36hc72bb7e_1
  - r-glue=1.4.2=r36hcfec24a_0
  - r-gplots=3.1.0=r36h6115d3f_0
  - r-gridextra=2.3=r36h6115d3f_0
  - r-gtable=0.3.0=r36h6115d3f_0
  - r-gtools=3.8.2=r36hcdcec82_1
  - r-hexbin=1.28.1=r36h31ca83e_2
  - r-highr=0.9=r36hc72bb7e_0
  - r-htmltools=0.5.1.1=r36h03ef668_0
  - r-htmlwidgets=1.5.3=r36hc72bb7e_0
  - r-httr=1.4.2=r36h6115d3f_0
  - r-hwriter=1.3.2=r36h6115d3f_0
  - r-isoband=0.2.4=r36h03ef668_0
  - r-jpeg=0.1_8.1=r36hcdcec82_1
  - r-jsonlite=1.7.2=r36hcfec24a_0
  - r-kernsmooth=2.23_18=r36h7679c2e_0
  - r-knitr=1.33=r36hc72bb7e_0
  - r-labeling=0.4.2=r36h142f84f_0
  - r-lambda.r=1.2.4=r36h6115d3f_1
  - r-later=1.2.0=r36h03ef668_0
  - r-lattice=0.20_44=r36hcfec24a_0
  - r-latticeextra=0.6_29=r36h6115d3f_1
  - r-lazyeval=0.2.2=r36h96ca727_0
  - r-lifecycle=1.0.0=r36hc72bb7e_0
  - r-magrittr=2.0.1=r36hcfec24a_1
  - r-markdown=1.1=r36hcfec24a_1
  - r-mass=7.3_54=r36hcfec24a_0
  - r-matrix=1.3_3=r36he454529_0
  - r-matrixstats=0.58.0=r36hcfec24a_0
  - r-mgcv=1.8_35=r36he454529_0
  - r-mime=0.10=r36hcfec24a_0
  - r-munsell=0.5.0=r36h6115d3f_0
  - r-nlme=3.1_150=r36h31ca83e_0
  - r-openssl=1.4.4=r36he36bf35_0
  - r-optparse=1.6.6=r36h6115d3f_1
  - r-pillar=1.6.1=r36hc72bb7e_0
  - r-pkgconfig=2.0.3=r36h6115d3f_1
  - r-pkgload=1.2.1=r36h03ef668_0
  - r-plotly=4.9.2.1=r36h6115d3f_1
  - r-plyr=1.8.6=r36h0357c0b_1
  - r-png=0.1_7=r36h96ca727_0
  - r-praise=1.0.0=r36h6115d3f_4
  - r-processx=3.5.2=r36hcfec24a_0
  - r-promises=1.2.0.1=r36h03ef668_0
  - r-ps=1.6.0=r36hcfec24a_0
  - r-purrr=0.3.4=r36hcfec24a_1
  - r-r6=2.5.0=r36hc72bb7e_0
  - r-rcolorbrewer=1.1_2=r36h6115d3f_0
  - r-rcpp=1.0.6=r36h03ef668_0
  - r-rcppparallel=5.0.2=r36h0357c0b_0
  - r-rcurl=1.98_1.2=r36hcdcec82_0
  - r-rematch2=2.1.2=r36h6115d3f_1
  - r-reshape=0.8.8=r36h6115d3f_0
  - r-reshape2=1.4.4=r36h0357c0b_1
  - r-rlang=0.4.11=r36hcfec24a_0
  - r-rmarkdown=2.5=r36h142f84f_0
  - r-rprojroot=2.0.2=r36hc72bb7e_0
  - r-rstudioapi=0.13=r36hc72bb7e_0
  - r-scales=1.1.1=r36h6115d3f_0
  - r-snow=0.4_3=r36h6115d3f_0
  - r-stringi=1.4.3=r36h29659fb_0
  - r-stringr=1.4.0=r36h6115d3f_0
  - r-sys=3.4=r36hcfec24a_0
  - r-testthat=3.0.2=r36h03ef668_0
  - r-tibble=3.1.2=r36hcfec24a_0
  - r-tidyr=1.1.3=r36h03ef668_0
  - r-tidyselect=1.1.1=r36hc72bb7e_0
  - r-tinytex=0.31=r36hc72bb7e_0
  - r-utf8=1.2.1=r36hcfec24a_0
  - r-vctrs=0.3.8=r36hcfec24a_1
  - r-viridislite=0.4.0=r36hc72bb7e_0
  - r-waldo=0.2.5=r36hc72bb7e_0
  - r-withr=2.4.2=r36hc72bb7e_0
  - r-xfun=0.23=r36hcfec24a_0
  - r-yaml=2.2.1=r36hcfec24a_1
  - re2=2022.06.01=h27087fc_0
  - readline=7.0=hf8c457e_1001
  - requests=2.28.1=pyhd8ed1ab_1
  - rsa=4.9=pyhd8ed1ab_0
  - ruffus=2.8.4=pyh864c0ab_1
  - s3transfer=0.6.0=pyhd8ed1ab_0
  - setuptools=59.8.0=py37h89c1867_1
  - six=1.16.0=pyh6c4a22f_0
  - sqlalchemy=1.4.42=py37h540881e_0
  - sqlite=3.33.0=h62c20be_0
  - sysroot_linux-64=2.12=he073ed8_15
  - time=1.8=h516909a_0
  - tk=8.6.12=h27826a3_0
  - tktable=2.10=hb7b940f_3
  - typing-extensions=4.4.0=hd8ed1ab_0
  - typing_extensions=4.4.0=pyha770c72_0
  - urllib3=1.26.12=py37h06a4308_0
  - wheel=0.38.4=pyhd8ed1ab_0
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - yarl=1.8.1=py37h5eee18b_0
  - zipp=3.10.0=pyhd8ed1ab_0
  - zlib=1.2.13=h166bdaf_4
  - zope.event=4.5.0=pyh9f0ad1d_0
  - zope.interface=5.5.0=py37h540881e_0
  - zstd=1.5.2=h6239696_4
  - pip:
    - ocms-16s==0.0.2

[nickilott]

Hi,

I wonder if this is to do with the length of reads - could you try and run it with trunclen: 150,150 in the pipeline.yml. I am in the process of getting things to run with the latest versions of everything so hopefully will have an update soon

[telatin]

Sorry, I forgot to mention that I then amended the pipeline.yaml with the 150,150 setting (and also tried the isolated command) but no luck

[nickilott]

Ok, thanks. I will look into it some more!

[nickilott]

Hi,

I'm sorry to be a pain but would you be able to try pulling the latest commit, removing your current ocms_16s conda env and trying to create an updated conda env that I have put in the repo?

You should be able to do

conda env create -f envs/environment_ocms_16s_v2.yml

from within the OCMS_16S directory. This has updated R, bioconductor etc and is working for me. It is a full export of my conda environment. Also I realised in the tutorial docs that I didn't point to the links for downloading the relevant files for dada2 to assign taxonomy that need to be included in the pipeline.yml. These are:

taxonomy_file: RefSeq-RDP16S_v2_May2018.fa.gz [download from here]

species_file: RefSeq-RDP_dada2_assignment_species.fa.gz [download from here]

[telatin]

When trying on the Linux box I was using for the test, I got:

Encountered problems while solving.
Problem: nothing provides requested libgfortran 5.0.0**
Problem: nothing provides icu 54.* needed by r-base-3.3.1-1

(using mamba 0.8.2, conda 4.9.2).

I will try on a fresh installation

[nickilott]

Thanks, it should be installing r-base-4.2.2 so it may be that you need to clear the pkg cache conda clean -all

[telatin]

True, but I made a fresh Linux VM with a fresh conda installation, same problem, unfortunately.
It's working fine on macOS though. Was it tested under Linux too?

UPDATE: Also under MacOS failed with

ERROR conda.core.link:_execute(730): An error occurred while installing package 'bioconda::bioconductor-genomeinfodbdata-1.2.9-r42hdfd78af_0'.

No idea what this is so will simply try again. I don't have a fresh MacBook, and I'm keener on testing under Linux, to be fair...

[nickilott]

It works on our HPC (CentOS 7) and on MacOS (on mine still running mojave). To be honest it hasn't been extensively tested across different OS...

[nickilott]

What version of linux are you running - will have a look into the issue

[telatin]

Intially Ubuntu 16.04.5 LTS (with pre-existing conda), then I tried Ubuntu 20.04 (fresh system and fresh conda).
I'm making a VM with CentOS now, will keep you posted

[nickilott]

awesome, thanks!

[telatin]

Also with CentOS Linux release 7.6.1810 (Core), while using mamba 0.15.3 / conda 4.12.0 (also happens using conda only):

Encountered problems while solving:
  - nothing provides requested libgfortran 5.0.0**
  - nothing provides icu 54.* needed by r-base-3.3.1-1

[nickilott]

Thanks, will look into it further. I'm still not sure why r-base-3.3.1 is being installed

[nickilott]

This could be something to do with mamba/conda. Similar thing seen here.

[telatin]

Would you be able to provide some guidance? Thanks!

[nickilott]

Hi, I came across this where there seemed to be similar issues installing anvio. Maybe you could try setting flexible channel priority as described

conda config --describe channel_priority
conda config --set channel_priority flexible

The only other thing would be to try and install using python virtualenv and pip although it would be beneficial to make sure this was working with conda/mamba.