Skip to content

Analysis refactor db (#2040) #2067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 191 commits into from
Jul 21, 2017
Merged

Analysis refactor db (#2040) #2067

merged 191 commits into from
Jul 21, 2017

Conversation

josenavas
Copy link
Contributor

PR to keep track of the things that need to be done to finish the analysis as plugins:

[ ] Refactor _build_biom_tables to merge based on commands/parameters/provenance.

@josenavas @antgonza
Analysis refactor db (#2040)
c92231c 
* New DB structure

* Adding python patch

* Adding a biom so we can actually execute the patch

* Fixing the patch to correctly transfer the information from the old structure to the new one

* Fixing patch

* Fixing patch and a few other bits to make the patch run successfully

* These files are no longer needed

* Droping analysis status table

* Linking the analysis with all the artifacts

* Fixing typo

* Fixing HTML and dbschema files

* Adding analyisis jobs

* Adding logging column to the analysis

* Addressing @antgonza's comments

* Taking into account non-phylogenetic metrics in beta diversity
@josenavas @ElDeveloper
Analysis refactor fix analysis (#2044)
9310326 
* New DB structure

* Adding python patch

* Adding a biom so we can actually execute the patch

* Fixing the patch to correctly transfer the information from the old structure to the new one

* Fixing patch

* Fixing patch and a few other bits to make the patch run successfully

* These files are no longer needed

* Removing unused code

* Droping analysis status table

* Linking the analysis with all the artifacts

* Fixing typo

* Fixing HTML and dbschema files

* Adding analyisis jobs

* Extending the artifact to work with the analysis

* Allowing multiomics datatype

* Adding private_job_submitter and modifying proc job handler to use it

* Adding logging column to the analysis

* Adding datatype to the analysis-processing job table

* Adding REST endpoint to access the analysis metadata

* Adding private jobs to plugin

* Fixing typo

* Fixing the processing jobs complete

* Addressing @antgonza's comments

* Taking into account non-phylogenetic metrics in beta diversity

* Addressing @antgonza's comments

* Addressing @ElDeveloper's comments
@josenavas @antgonza
Remove old job code (#2045)
23abbdb 
* New DB structure

* Adding python patch

* Adding a biom so we can actually execute the patch

* Fixing the patch to correctly transfer the information from the old structure to the new one

* Fixing patch

* Fixing patch and a few other bits to make the patch run successfully

* These files are no longer needed

* Removing unused code

* Droping analysis status table

* Linking the analysis with all the artifacts

* Fixing typo

* Fixing HTML and dbschema files

* Adding analyisis jobs

* Extending the artifact to work with the analysis

* Allowing multiomics datatype

* Adding private_job_submitter and modifying proc job handler to use it

* Adding logging column to the analysis

* Adding datatype to the analysis-processing job table

* Adding REST endpoint to access the analysis metadata

* Adding private jobs to plugin

* Fixing typo

* Fixing the processing jobs complete

* Removing the old job code

* Oops removed the wrong file

* Addressing @antgonza's comments

* Taking into account non-phylogenetic metrics in beta diversity

* Addressing @antgonza's comments
@josenavas @antgonza
Fixes Qiita db (#2046)
e094701 
* New DB structure

* Adding python patch

* Adding a biom so we can actually execute the patch

* Fixing the patch to correctly transfer the information from the old structure to the new one

* Fixing patch

* Fixing patch and a few other bits to make the patch run successfully

* These files are no longer needed

* Removing unused code

* Droping analysis status table

* Linking the analysis with all the artifacts

* Fixing typo

* Fixing HTML and dbschema files

* Adding analyisis jobs

* Extending the artifact to work with the analysis

* Allowing multiomics datatype

* Adding private_job_submitter and modifying proc job handler to use it

* Adding logging column to the analysis

* Adding datatype to the analysis-processing job table

* Adding REST endpoint to access the analysis metadata

* Adding private jobs to plugin

* Fixing typo

* Fixing the processing jobs complete

* Removing the old job code

* Oops removed the wrong file

* Removing QiitaStatusObject because it is not used

* fixing metautil

* Fixing porntal, setup and sql tests

* Fixing user and util

* Fixing qiita_db

* Addressing @antgonza's comments

* Taking into account non-phylogenetic metrics in beta diversity

* Addressing @antgonza's comments
@josenavas @antgonza
Analysis refactor master (#2070)
3fdb802 
* fix #1505

* improving some GUI stuff

* improving some GUI stuff - missing lines

* addressing all comments

* ready for review

* New DB structure

* Adding python patch

* Adding a biom so we can actually execute the patch

* Fixing the patch to correctly transfer the information from the old structure to the new one

* Fixing patch

* Fixing patch and a few other bits to make the patch run successfully

* These files are no longer needed

* Removing unused code

* Droping analysis status table

* Linking the analysis with all the artifacts

* Fixing typo

* Fixing HTML and dbschema files

* Adding analyisis jobs

* Extending the artifact to work with the analysis

* Allowing multiomics datatype

* Adding private_job_submitter and modifying proc job handler to use it

* Adding logging column to the analysis

* Adding datatype to the analysis-processing job table

* Adding REST endpoint to access the analysis metadata

* Adding private jobs to plugin

* Fixing typo

* Fixing the processing jobs complete

* Removing the old job code

* Oops removed the wrong file

* Removing QiitaStatusObject because it is not used

* fixing metautil

* Fixing porntal, setup and sql tests

* Fixing user and util

* Fixing qiita_db

* fix #1987

* Updating to master

* Addressing @antgonza's comments

* initial commit

* requested changes

* fix filter job list

* Fixing server cert (#2051)

* fix get_studies

* flake8

* fix #503

* fix #2010

* fix #1913

* fix errors

* addressing @josenavas comment

* flake8

* fix #1010

* fix #1066 (#2058)

* addressing @josenavas comments

* fix #1961

* fix #1837

* Automatic jobs & new stats (#2057)

* fix #814, fix #1636

* fixing error in test-env

* fixing stats.html call

* adding img

* addressing @josenavas comments

* rm for loops

* addresssing @ElDeveloper comments

* generalizing this functionality

* fix #1816

* Taking into account non-phylogenetic metrics in beta diversity

* fix #1959

* Addressing @antgonza's comments

* addressing @josenavas comments

* addressing @josenavas comments

* fixing error

* fixed?

* addressing @josenavas comments

* Actually fixing merge conflicts

* addressing @wasade comments

* fix flake8

* generate biom and metadata release (#2066)

* initial commit

* adding portal

* addressing @josenavas comments

* pid -> qiita_artifact_id

* addressing @josenavas comments

* addressing @ElDeveloper comments

* rm 50.sql

* database changes to fix 969

* adding delete

* addressing @josenavas comments

* addressing @ElDeveloper comments

* duh!
@josenavas
Fixing merge conflicts with master
1ab4e3b
@josenavas @antgonza
Sync-ing with master (again) (#2073)
1377640 
* fix #1505

* improving some GUI stuff

* improving some GUI stuff - missing lines

* addressing all comments

* ready for review

* fix #1987

* initial commit

* requested changes

* fix filter job list

* Fixing server cert (#2051)

* fix get_studies

* flake8

* fix #503

* fix #2010

* fix #1913

* fix errors

* addressing @josenavas comment

* flake8

* fix #1010

* fix #1066 (#2058)

* addressing @josenavas comments

* fix #1961

* fix #1837

* Automatic jobs & new stats (#2057)

* fix #814, fix #1636

* fixing error in test-env

* fixing stats.html call

* adding img

* addressing @josenavas comments

* rm for loops

* addresssing @ElDeveloper comments

* generalizing this functionality

* fix #1816

* fix #1959

* addressing @josenavas comments

* addressing @josenavas comments

* fixing error

* fixed?

* addressing @josenavas comments

* addressing @wasade comments

* fix flake8

* generate biom and metadata release (#2066)

* initial commit

* adding portal

* addressing @josenavas comments

* pid -> qiita_artifact_id

* addressing @josenavas comments

* addressing @ElDeveloper comments

* rm 50.sql

* database changes to fix 969

* adding delete

* addressing @josenavas comments

* addressing @ElDeveloper comments

* duh!

* fix generate_biom_and_metadata_release (#2072)

* fix generate_biom_and_metadata_release

* addressing @ElDeveloper comment
@josenavas
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
1e8332e 
…into analysis-refactor
@josenavas
Removing qiita ware code that will not be used anymore
cb67d3d
@josenavas @antgonza
Fix qiita ware (#2074)
ee170a0 
* fix #1505

* improving some GUI stuff

* improving some GUI stuff - missing lines

* addressing all comments

* ready for review

* fix #1987

* initial commit

* requested changes

* fix filter job list

* Fixing server cert (#2051)

* fix get_studies

* flake8

* fix #503

* fix #2010

* fix #1913

* fix errors

* addressing @josenavas comment

* flake8

* fix #1010

* fix #1066 (#2058)

* addressing @josenavas comments

* fix #1961

* fix #1837

* Automatic jobs & new stats (#2057)

* fix #814, fix #1636

* fixing error in test-env

* fixing stats.html call

* adding img

* addressing @josenavas comments

* rm for loops

* addresssing @ElDeveloper comments

* generalizing this functionality

* fix #1816

* fix #1959

* addressing @josenavas comments

* addressing @josenavas comments

* fixing error

* fixed?

* addressing @josenavas comments

* addressing @wasade comments

* fix flake8

* generate biom and metadata release (#2066)

* initial commit

* adding portal

* addressing @josenavas comments

* pid -> qiita_artifact_id

* addressing @josenavas comments

* addressing @ElDeveloper comments

* rm 50.sql

* database changes to fix 969

* adding delete

* addressing @josenavas comments

* addressing @ElDeveloper comments

* duh!

* fix generate_biom_and_metadata_release (#2072)

* fix generate_biom_and_metadata_release

* addressing @ElDeveloper comment

* Removing qiita ware code that will not be used anymore
@josenavas
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
5a5127d 
…into analysis-refactor-gui-part1
@josenavas
Organizing the handlers and new analysis description page
0033480
@josenavas
Connecting the analysis creation and making interface responsive
3809ad5
@josenavas
Addressing @antgonza's comments
067f14f
@josenavas
Solving merge conflicts
cf4862d
@josenavas
Initial artifact GUI refactor
3b07151
@josenavas
Removing unused code
a6595a9
@josenavas
Merge branch 'analysis-refactor-gui-part2' into analysis-refactor-gui…
6343b49 
…-part3
@josenavas @antgonza
Analysis refactor GUI part1 (#2076)
a083e23 
* fix #1505

* improving some GUI stuff

* improving some GUI stuff - missing lines

* addressing all comments

* ready for review

* fix #1987

* initial commit

* requested changes

* fix filter job list

* Fixing server cert (#2051)

* fix get_studies

* flake8

* fix #503

* fix #2010

* fix #1913

* fix errors

* addressing @josenavas comment

* flake8

* fix #1010

* fix #1066 (#2058)

* addressing @josenavas comments

* fix #1961

* fix #1837

* Automatic jobs & new stats (#2057)

* fix #814, fix #1636

* fixing error in test-env

* fixing stats.html call

* adding img

* addressing @josenavas comments

* rm for loops

* addresssing @ElDeveloper comments

* generalizing this functionality

* fix #1816

* fix #1959

* addressing @josenavas comments

* addressing @josenavas comments

* fixing error

* fixed?

* addressing @josenavas comments

* addressing @wasade comments

* fix flake8

* generate biom and metadata release (#2066)

* initial commit

* adding portal

* addressing @josenavas comments

* pid -> qiita_artifact_id

* addressing @josenavas comments

* addressing @ElDeveloper comments

* rm 50.sql

* database changes to fix 969

* adding delete

* addressing @josenavas comments

* addressing @ElDeveloper comments

* duh!

* fix generate_biom_and_metadata_release (#2072)

* fix generate_biom_and_metadata_release

* addressing @ElDeveloper comment

* Removing qiita ware code that will not be used anymore

* Organizing the handlers and new analysis description page

* Addressing @antgonza's comments
@josenavas
Adding can_edit call to the analysis
f731768
@josenavas
Fixing artifact rest API since not all artifacts have study
7542658
@josenavas
Adding can_be_publicized call to analysis
e0180e8
@josenavas
Adding QiitaHTTPError to handle errors gracefully
f55ca5c
@josenavas
Adding safe_execution contextmanager
1fa4b19
@josenavas
Fixing typo
b61ae87
@josenavas
Adding qiita test checker
bb68303
@josenavas
Adapting some artifact handlers
b31a025
@josenavas
Fixing merge conflicts
378d7ff
@josenavas
Merge branch 'analysis-refactor-gui-part2' into analysis-refactor-gui…
444da08 
…-part3
@josenavas
Abstracting the graph reloading and adding some documentation
f6b4c46
josenavas and others added 28 commits June 29, 2017 16:08
@josenavas @antgonza
Fix artifact type creation (#2158)
080be0f 
* Changing the HTML setter to add support dirs

* Extending the patch operation

* Fixing remaining calls to html_summary_fp set

* Fixing test

* Deleting unused function

* Removing unused import

* Fixing tests

* Fixing failing test

* Creating the mountpoint of the new type

* Cleaning up patch and fixing bug

* Fixing tests

* Adding analysis to the list of acceptable parameter types

* Adding the provenance parameter automatically
@josenavas @antgonza
Adding qzv fileapth type (#2159)
5572c0f 
* Adding qzv fileapth type

* Fixing test
@antgonza
fixing tests
86f7bbc
@antgonza
adding ORDER
983d606
@antgonza
to debug
cca49c3
@antgonza
splitting the plugin registration
581bbcd
@antgonza
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
37092a9 
…into improving-artifact-info
@josenavas
Merge pull request #2160 from antgonza/split-plugin-start
7327507 
splitting the plugin registration
@antgonza
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
2112607 
…into improving-artifact-info
@josenavas @antgonza
Analysis refactor misc (#2161)
85f6ec7 
* Fixing race condition

* Removing unused code and fixing typo

* Hiding job step if success

* Showing loading gif and fixing typo
@antgonza
update upstream
c1e468b
@antgonza
fix tests
26e4734
@antgonza
to debug
98b293d
@antgonza @ElDeveloper
improving delete speed (#2165)
21f8858 
* improving delete speed

* more sleep!
@antgonza
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
561e837 
…into improving-artifact-info
@antgonza @ElDeveloper
fix artifact summary issue (#2163)
4c91753 
* fix artifact summary issue

* sleep
@antgonza
Merge branch 'analysis-refactor' of https://github.com/biocore/qiita …
65f741a 
…into improving-artifact-info
@antgonza
fixing errors 1
03e9f55
@antgonza
missing ]
2d53fea
@antgonza
fix error
44ab61f
@antgonza
barcode_type 8, defaults
b114500
@josenavas
Merge pull request #2149 from antgonza/improving-artifact-info
ab0e2f2 
Improving artifact info and study listing
@antgonza @ElDeveloper
WIP: using new info to merge bioms (#2167)
9a378bc 
* using new info to merge bioms

* fix errors

* flake8
@antgonza @ElDeveloper
fix 55.py (#2168)
2294c3f 
* fix 55.py

* fixing small bug in test env

* addressing @ElDeveloper comments

* config_test.cfg

* rm /test_data/ from folders

* fixing

* releases

* working_dir/releases

* working_dir/
@antgonza @ElDeveloper
Fix analysis artifact name (#2169)
c8042c7 
* fix analysis artifact name

* fix analysis artifact name

* params -> md5

* ignoring params and target subfragment

* flake8

* remonving parent_processing
@antgonza @ElDeveloper
Improving analysis listing (#2170)
fe7879b 
* improving analysis listing

* addressing @ElDeveloper comments
@antgonza @ElDeveloper
Improving analysis listing b (#2175)
b0891a1 
* improving analysis listing

* addressing @ElDeveloper comments

* adding list artifacts

* changing names

* fixing . -> ,

* only_biom

* actually using only_biom

* sleep

* moving sleep

* import sleep

* sleep(0.8)

* sleep(1)

* addressing @ElDeveloper comments

* flake8

* is_qiita_job

* job_info -> ji

* addressing @ElDeveloper comments
@antgonza @ElDeveloper
Fix branch (#2176)
e31e661 
* improving analysis listing

* addressing @ElDeveloper comments

* adding list artifacts

* changing names

* fixing . -> ,

* only_biom

* actually using only_biom

* sleep

* moving sleep

* import sleep

* sleep(0.8)

* sleep(1)

* addressing @ElDeveloper comments

* flake8

* is_qiita_job

* job_info -> ji

* addressing @ElDeveloper comments

* fixing branch
@antgonza antgonza merged commit 8ad7064 into master Jul 21, 2017
@antgonza antgonza deleted the analysis-refactor branch August 8, 2017 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@josenavas @coveralls @codecov-io @antgonza