fix #2868 #3028

antgonza · 2020-09-03T00:22:09Z

No description provided.

codecov-commenter · 2020-09-03T00:42:53Z

Codecov Report

Merging #3028 into dev will increase coverage by 0.01%.
The diff coverage is 91.85%.

@@            Coverage Diff             @@
##              dev    #3028      +/-   ##
==========================================
+ Coverage   94.95%   94.97%   +0.01%     
==========================================
  Files          74       74              
  Lines       14291    14177     -114     
==========================================
- Hits        13570    13464     -106     
+ Misses        721      713       -8

Impacted Files	Coverage Δ
qiita_db/download_link.py	`33.33% <0.00%> (+3.60%)`	⬆️
qiita_db/analysis.py	`96.80% <72.72%> (+0.43%)`	⬆️
qiita_db/artifact.py	`98.26% <87.50%> (-0.02%)`	⬇️
qiita_db/study.py	`94.02% <87.50%> (+0.17%)`	⬆️
qiita_db/logger.py	`98.11% <100.00%> (-0.14%)`	⬇️
qiita_db/meta_util.py	`88.62% <100.00%> (-0.08%)`	⬇️
...ita_db/metadata_template/base_metadata_template.py	`96.33% <100.00%> (-0.02%)`	⬇️
qiita_db/metadata_template/prep_template.py	`98.03% <100.00%> (-0.05%)`	⬇️
qiita_db/ontology.py	`100.00% <100.00%> (ø)`
qiita_db/portal.py	`99.29% <100.00%> (-0.01%)`	⬇️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f13ac2...93c48cb. Read the comment docs.

antgonza · 2020-09-03T16:43:23Z

This is ready for review. There are lots of changes but there is only one main change here, which is then propagated to all the rest of the changes to make the tests more DRY.

ElDeveloper

Looks good, just a couple of questions.

qiita_db/test/test_meta_util.py

qiita_db/test/test_study.py

ElDeveloper · 2020-09-03T23:55:17Z

qiita_db/sql_connection.py

@@ -493,6 +493,24 @@ def add_post_rollback_func(self, func, *args, **kwargs):
 TRNADMIN = Transaction(admin=True)


+def encapsulated_query(sql, parameters=None):


Name is fine but I think it would be nice to make it clear that queries are executed as transactions? Up to your discretion. What about execute_as_transaction.

Yeah, selected single_query first, then changed it to encapsulted_query but still not happy. execute_as_transaction is a decorator we hace; thus, gonna change to perform_as_transaction.

antgonza

Thank you @ElDeveloper

antgonza · 2020-09-04T13:42:18Z

qiita_db/sql_connection.py

@@ -493,6 +493,24 @@ def add_post_rollback_func(self, func, *args, **kwargs):
 TRNADMIN = Transaction(admin=True)


+def encapsulated_query(sql, parameters=None):


Yeah, selected single_query first, then changed it to encapsulted_query but still not happy. execute_as_transaction is a decorator we hace; thus, gonna change to perform_as_transaction.

qiita_db/test/test_meta_util.py

ElDeveloper · 2020-09-04T18:48:00Z

Thanks @antgonza would you mind fixing the conflicts?

antgonza · 2020-09-04T18:51:12Z

Done, didn't see them before ... thank you!

ElDeveloper · 2020-09-04T22:18:45Z

Thanks!

@ElDeveloper

* inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

@ElDeveloper

* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * fix #3036 Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

@ElDeveloper

…3040) * Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * adding job submit ENVIRONMENT and minor improvement to runWorkflow * fix test Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

@ElDeveloper

* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * fix #2920 * fix test Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

@ElDeveloper

* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * rm create_qiime_mapping_file * fixing some tests * fixing more tests * fix even more tests * rm npt.assert_warns * qiime-map -> sample-file * update not_merged_samples.txt * adding @ElDeveloper changes Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

@ElDeveloper

* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Antonio Gonzalez <antgonza@gmail.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> * adding qiita.study autoloaded column * cleaning tests * fix redbiom test * avoiding clog submissions * Apply suggestions from code review Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu> Co-authored-by: Daniel McDonald <danielmcdonald@ucsd.edu> Co-authored-by: Mirte Kuijpers <67341505+mcmk3@users.noreply.github.com> Co-authored-by: Yoshiki Vázquez Baeza <yoshiki@ucsd.edu>

fix qiita-spots#2868

9f5ba45

antgonza added 7 commits September 2, 2020 18:59

2nd round

6831e5c

fix errors

eb7b52e

more changes

99a7fd1

fix errors

1de5044

fix ProcessingJobTest

82d87c1

fix PY_PATCH

28eda5c

add missing TRN.add

556dc9c

antgonza requested a review from ElDeveloper September 3, 2020 16:43

antgonza changed the title ~~WIP: fix #2868~~ fix #2868 Sep 3, 2020

ElDeveloper approved these changes Sep 3, 2020

View reviewed changes

antgonza commented Sep 4, 2020

View reviewed changes

encapsulated_query -> perform_as_transaction

0d1ffb7

fix conflicts

93c48cb

ElDeveloper merged commit 6d30da1 into qiita-spots:dev Sep 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix #2868 #3028

fix #2868 #3028

Uh oh!

antgonza commented Sep 3, 2020

Uh oh!

codecov-commenter commented Sep 3, 2020 •

edited

Loading

Uh oh!

antgonza commented Sep 3, 2020

Uh oh!

ElDeveloper left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ElDeveloper Sep 3, 2020

Uh oh!

antgonza Sep 4, 2020

Uh oh!

ElDeveloper Sep 4, 2020

Uh oh!

antgonza left a comment

Uh oh!

antgonza Sep 4, 2020

Uh oh!

Uh oh!

ElDeveloper commented Sep 4, 2020

Uh oh!

antgonza commented Sep 4, 2020

Uh oh!

ElDeveloper commented Sep 4, 2020

Uh oh!

Uh oh!

		@@ -493,6 +493,24 @@ def add_post_rollback_func(self, func, args, *kwargs):
		TRNADMIN = Transaction(admin=True)


		def encapsulated_query(sql, parameters=None):

fix #2868 #3028

fix #2868 #3028

Uh oh!

Conversation

antgonza commented Sep 3, 2020

Uh oh!

codecov-commenter commented Sep 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

antgonza commented Sep 3, 2020

Uh oh!

ElDeveloper left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ElDeveloper Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

antgonza Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

ElDeveloper Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

antgonza left a comment

Choose a reason for hiding this comment

Uh oh!

antgonza Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ElDeveloper commented Sep 4, 2020

Uh oh!

antgonza commented Sep 4, 2020

Uh oh!

ElDeveloper commented Sep 4, 2020

Uh oh!

Uh oh!

codecov-commenter commented Sep 3, 2020 •

edited

Loading