-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Missing file in output folder of mmvec paired-omics command #183
Comments
Hi @natalie-melendez its hard to see why the model_stats.qza wasn't generated. Do you run with the verbose flag? How long did it run? What command did you use? How many epoches was run? What summary interval was used? |
Hi @mortonjt! Initially I did not run with the verbose flag. I ran it again using the verbose flag and this is what I got: (qiime2-2020.6) megl5@x86_64-apple-darwin13 ~ % qiime mmvec paired-omics --i-microbes 16S-no-sings-filtered-rarified-N30-non-Bifido-glioma.qza --i-metabolites untargeted-metabolites-RGDOXDH-time-series-N30-glioma.qza --p-summary-interval 1 --output-dir mmvec-model-summary-N30-non-Bifido-glioma --verbose See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return constructor(mat, index=index, columns=columns)
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. sparse_index=BlockIndex(N, blocs, blens),
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. default_kind=self._default_kind,
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return klass(values, index=self.index, name=items, fastpath=True) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_arrays, index=index, columns=columns).finalize(
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_values, index=self.index, name=self.name) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. default_fill_value=self.default_fill_value, See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self)
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. result = self._constructor(new_data).finalize(self) 2024-11-22 14:39:48.905339: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:91: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:118: Normal.init (from tensorflow.python.ops.distributions.normal) is deprecated and will be removed after 2019-01-01. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:185: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:189: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:191: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:196: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/tensorflow_core/python/ops/clip_ops.py:301: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:243: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead. 100%|███████████████████████████████████████████████████████████| 3662/3662 [00:04<00:00, 824.97it/s] I'm fairly new with mmvec. I don't understand your question in reference to epoches. What is epoches? Also, I used the summary interval at 1 (same value as in the original mmvec github), however truthfully I do not know what this is for. Also, when I run the command, in addition to creating the output folder the command creates another folder: |
Try running the following code with 10000 epochs -- it'll make it run longer (and potentially make it more accurate).
|
I ran it with 10000 epochs, however, it still only generates 2 files. (qiime2-2020.6) megl5@x86_64-apple-darwin13 ~ % qiime mmvec paired-omics --i-microbes 16S-no-sings-filtered-rarified-N30-non-Bifido-glioma.qza --i-metabolites untargeted-metabolites-RGDOXDH-time-series-N30-glioma.qza --p-summary-interval 1 --output-dir mmvec-model-summary-N30-non-Bifido-glioma --verbose --p-epochs 10000 See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return constructor(mat, index=index, columns=columns)
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. sparse_index=BlockIndex(N, blocs, blens),
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. default_kind=self._default_kind,
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return klass(values, index=self.index, name=items, fastpath=True) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_arrays, index=index, columns=columns).finalize(
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_values, index=self.index, name=self.name) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. default_fill_value=self.default_fill_value, See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self)
See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. return self._constructor(new_data).finalize(self) See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more. result = self._constructor(new_data).finalize(self) 2024-11-22 15:08:14.223568: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:91: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:118: Normal.init (from tensorflow.python.ops.distributions.normal) is deprecated and will be removed after 2019-01-01. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:185: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:189: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:191: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:196: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/tensorflow_core/python/ops/clip_ops.py:301: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. WARNING:tensorflow:From /opt/miniconda3/envs/qiime2-2020.6/lib/python3.6/site-packages/mmvec/multimodal.py:243: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead. 100%|███████████████████████████████████████████████████████| 363400/363400 [08:39<00:00, 699.26it/s] |
H @natalie-melendez I'm not sure why that file isn't being generated. But it is not the first time this error have been reported https://forum.qiime2.org/t/no-model-stats-qza-file/29044/16 I'd be curious to hear if the non-qiime2 version generated tensor flow results. When you run |
Hi @mortonjt. Interestingly, yesterday I was looking at how to generate the heatmap with mmvec and I saw it needed the rank.qza file which I don't have and didn't know how to obtain it. Seeing the forum you have shared, I noticed that those files should have been generated inside the summary output folder during the paired-omics command? I had a lot of issues downloading either qiime2-2020.6 or the mmvec standalone. Currently, I do not have the non-qiime2 version but I will try to download it and see if by using the non-qiime2 version I can generate those files. When you say to run tensorboard --logdir ., were do I do that? I apologize if this is a simple question, I am new with this setup. Thank you! |
Hi @mortonjt! I tried installing the non-qiime2 mmvec version. However, even though it all seems like it installed properly and I am able to activate the environment, it gives me a traceback error. I'm seeing that the tensorflow-estimator package being installed is at version 2.6.0, while tensorflow and tensorflow-base is at 1.15.0. Should I downgrade the tensorflow-estimator? (base) megl5@MEGLs-4-iMac ~ % conda create -n mmvec_env mamba python=3.7 -c conda-forge
Package Planenvironment location: /opt/miniconda3/envs/mmvec_env added / updated specs: The following NEW packages will be INSTALLED: bzip2 conda-forge/osx-64::bzip2-1.0.8-hfdf4475_7 Proceed ([y]/n)? y Downloading and Extracting Packages: Preparing transaction: done To activate this environment, use$ conda activate mmvec_envTo deactivate an active environment, use$ conda deactivate(base) megl5@MEGLs-4-iMac ~ % warning libmamba Please make sure you understand Anaconda Terms of Services. warning libmamba See: https://legal.anaconda.com/policies/en/ warning libmamba Please make sure you understand Anaconda Terms of Services. warning libmamba See: https://legal.anaconda.com/policies/en/ warning libmamba Please make sure you understand Anaconda Terms of Services. warning libmamba See: https://legal.anaconda.com/policies/en/ warning libmamba Please make sure you understand Anaconda Terms of Services. warning libmamba See: https://legal.anaconda.com/policies/en/ Pinned packages:
Transaction Prefix: /opt/miniconda3/envs/mmvec_env Updating specs:
Package Version Build Channel Size
Change:
Downgrade:
Summary: Install: 136 packages Total download: 265MB ────────────────────────────────────────────────────────────────────────────────────────────── Confirm changes: [Y/n] y Transaction starting Transaction finished (mmvec_env) megl5@MEGLs-4-iMac ~ % |
Hi,
I recently starting using mmvec for microbial-metabolome integration and when performing the mmvec paired-omics command on qiime2-2020.6 it only generated two files: conditional_biplot.qza and conditionals.qza. However, it did not generate the model_stats.qza. How can I resolve this issue?
Thank you!
The text was updated successfully, but these errors were encountered: