Make sure that PrepTemplate and SampleTemplate does not have columns in common

[josenavas]

as part of the metadata validation.

[ackermag]

sample_name was always a duplicate field - will it be in sample or prep?

[rob-knight]

Don't you need that info in both so you can join them?

On Oct 19, 2014, at 1:09 PM, ackermag <notifications@github.commailto:notifications@github.com> wrote:

sample_name was always a duplicate field - will it be in sample or prep?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/388#issuecomment-59660950.

[ackermag]

It makes curation easier - we generally get sample metadata before we get barcode mapping to sample names, lists do not match EVER. Is there a way to line up sample and map with barcodes computationally in the upload process so that mismatches are identified and can be corrected?

[ackermag]

I believe I responded - let me know if I didn't

On Sun, Oct 19, 2014 at 1:17 PM, Rob Knight notifications@github.com
wrote:

Don't you need that info in both so you can join them?

On Oct 19, 2014, at 1:09 PM, ackermag <notifications@github.com<mailto:
notifications@github.com>> wrote:

sample_name was always a duplicate field - will it be in sample or prep?

—
Reply to this email directly or view it on GitHub<
https://github.com/biocore/qiita/issues/388#issuecomment-59660950>.

—
Reply to this email directly or view it on GitHub
#388 (comment).

Gail Ackermann
Knight Lab UCB 596
University of Colorado
Boulder

[josenavas]

@ackermag sorry I was not clear enough.

sample_name must be on both of them, but not metadata columns.

Thanks

[antgonza]

I think this should be easy to implement if:

1. We check that sample template doesn't have any of the prep template required fields, and
2. We check that the prep doesn't have any overlapping columns names
   Note that we need sample_name to relate both templates.

However, is not clear for me what should we do with the columns (1151) of the studies (224) that already have this problem. From a quick look it seems that 99% of them are fields that shouldn't be in the prep template but they are.

The SQLs I used to examine this, just for reference:

select st.study_id, st.column_name, pt.prep_template_id from qiita.study_sample_columns as st join qiita.study_prep_template as stpt on st.study_id=stpt.study_id left join qiita.prep_columns as pt on stpt.prep_template_id=pt.prep_template_id where st.column_name=pt.column_name;

select distinct st.study_id from qiita.study_sample_columns as st join qiita.study_prep_template as stpt on st.study_id=stpt.study_id left join qiita.prep_columns as pt on stpt.prep_template_id=pt.prep_template_id where st.column_name=pt.column_name;

[josenavas]

It may be worth double-checking, but I think this is no longer an issue since we are adding a suffix to the prep template columns when we generate the qiime mapping file, which was where the problem appeared.

[antgonza]

Got it, didn't know that was the issue. However, I think it will be nice to not have duplicated information ...

[josenavas]

Not sure, but is possible that the information is not duplicated, is just that the columns have the same name (probably a bad design but can happen...)

[antgonza]

Again, possible but not desirable. Some of the column names are: year, collection_timestamp, center_name, watertimebarn; which clearly should be sample and not prep metadata.

[antgonza]

Turns out that this is more important than expected: if the sample info file has barcode/barcodesequence, it will take precedence over the prep info and will be reused. Moving to Alpha 0.3.