Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulkloader Column Names - Reprise #6171

Closed
Jegelewicz opened this issue Apr 20, 2023 · 28 comments
Closed

Bulkloader Column Names - Reprise #6171

Jegelewicz opened this issue Apr 20, 2023 · 28 comments
Assignees
Labels
Enhancement I think this would make Arctos even awesomer! Function-DataEntry/Bulkloading Priority - Wildfire Potential ignore this at everyone's peril, may smolder for now ...

Comments

@Jegelewicz
Copy link
Member

Jegelewicz commented Apr 20, 2023

If you want background see #6103 and #5193 (comment)

In an effort to make the bulkloader better while it is being rebuilt, the Code Table Admins went through the currently proposed column headers and revised them for consistency and logical grouping. You can review our efforts here: Bulkloader Rebuild

The first column includes the headers from the current bulkloader, the middle (hidden) column was Dusty's original proposal and the last column is the Code Table Admins' proposal. This is expected to be our bulkloader for about a five year time frame, so please review this and record any objections or provide your support for the revision in this issue.

Note there will be many more columns in the actual bulkloader as some sections will be duplicated. For more information see

Current table is always https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader


START DLM bits

temp_maybe_new_bulk_cols(2).csv

temp_maybe_new_bulkloader(11).csv

Column Count:1137

lastedit: #6255

lastedit: associated_species

END DLM bits


Summary: https://docs.google.com/spreadsheets/d/1ssNP_jAiOok7TYIPq8b-OKcLCw9NngtCElPwbS6ajFo/edit#gid=0

1136

Barring a block by #6170 (which you should also review!), we plan to adopt the revised bulkloader column headers at the AWG Issues Meeting on May 4, 2023

@ewommack could you please send an email to the AWG with the text of this issue? Thanks!

@Jegelewicz Jegelewicz added Function-DataEntry/Bulkloading Enhancement I think this would make Arctos even awesomer! Priority - Wildfire Potential ignore this at everyone's peril, may smolder for now ... labels Apr 20, 2023
@ewommack
Copy link

@Jegelewicz - as a reply to the last bulkloader email?

@lin-fred - just an FYI

@Jegelewicz
Copy link
Member Author

as a reply to the last bulkloader email?

I do not know what was in that email. I don't think it matters if it is a reply or a new email though.

@ewommack
Copy link

Okey dokey

@ewommack
Copy link

And sent. Text below:
Arctos members,

Arctos needs your help.

In an effort to make the bulkloader better while it is being rebuilt, the Code Table Admins went through the currently proposed column headers and revised them for consistency and logical grouping. You can review our efforts here: Bulkloader Rebuild

The first column includes the headers from the current bulkloader, the middle column is Dusty's current proposal and the last column is the Code Table Admins' revision. This is expected to be our bulkloader for about a five year time frame, so please review this and record any objections or provide your support for the revision in this issue.

And for additional background please see #6103 and #5193 (comment)

Barring a block by #6170 (which you should also review!), we plan to adopt the revised bulkloader column headers at the AWG Issues Meeting on May 4, 2023

--

@dustymc
Copy link
Contributor

dustymc commented Apr 24, 2023

I think this probably should not be closed, and I think it should be communicated to the whole community rather than just the working group??

@Jegelewicz Jegelewicz reopened this Apr 24, 2023
@Jegelewicz
Copy link
Member Author

I think it should be communicated to the whole community

The only way I know to do this is with a banner - we did not discuss that - should we add one? @ArctosDB/arctos-working-group-officers

@dustymc
Copy link
Contributor

dustymc commented Apr 24, 2023

I certainly don't know what nor how, but if something emerges please add it to https://github.com/ArctosDB/internal/issues/258#issuecomment-1515440177 - the mechanics of (2) are still pretty foggy from here, this is something that will/may require changes from near everybody, and this is something that will not be easily un-done.

@campmlc
Copy link

campmlc commented Apr 24, 2023

I agree with a banner notification.

@Jegelewicz
Copy link
Member Author

New Banner

Notice: A rebuild of the bulkloader is planned soon. For more details see this Github Issue. We plan to vote to adopt the new bulkloader at the AWG Issues Meeting on May 4, 2023. Please post any objection, request, or agreement in the Issue before May 2, 2023.

Any edits before I post?

@campmlc
Copy link

campmlc commented Apr 24, 2023 via email

@dustymc
Copy link
Contributor

dustymc commented Apr 24, 2023

Your link is going to a comment.

I think the proposed column names should be more explicit.

"My" proposal (its not??) can be removed.

It should be explained (somehow) that the linked sheet isn't complete, eg there are some ridiculous number of various things in the current proposal, resulting in over a thousand columns.

The implications of changing all the names should probably be summarized somewhere.

What precisely does "scheduled for final approval and implementation" mean? (For both me and someone who might want a change in a couple weeks, please.)

"We plan to vote to adopt " - who's "we"? (Maybe that's known from other stuff?)

@Jegelewicz
Copy link
Member Author

Notice: A planned rebuild of the bulkloader is scheduled for final approval at the AWG Issues Meeting on May 4, 2023. For more details see #6171. Please post any objection, request, or agreement in the Issue before May 2, 2023.

@Jegelewicz
Copy link
Member Author

I think the proposed column names should be more explicit.

Ideas accepted

"My" proposal (its not??) can be removed.

I want this there for the purpose of evolution - if everyone else thinks it can go away, then I can delete it - I have hidden it now.

It should be explained (somehow) that the linked sheet isn't complete, eg there are some ridiculous number of various things in the current proposal, resulting in over a thousand columns.

That is in #5193 and I placed a summary in the first comment for those who won't review the background.

The implications of changing all the names should probably be summarized somewhere.

And those are?

What precisely does "scheduled for final approval and implementation" mean? (For both me and someone who might want a change in a couple weeks, please.)

Removed implementation - only final approval is on the table.

"We plan to vote to adopt " - who's "we"? (Maybe that's known from other stuff?)

Changed - shouldn't be a problem now?

@dustymc
Copy link
Contributor

dustymc commented Apr 24, 2023

That is in #5193

That should be closed; this is no longer additive and has become something very different.

And those are?

  • Anything that uses or references "old" column names will no longer function.
  • I (and the dev environment) will be unavailable for other tasks for ~a month
  • Beyond that core - you tell me.
    • Do we need to update component loaders?
    • Do we need to update any other part of the UI?
    • Does the pre-bulkloader need to survive this?
    • Can eg Feature Request - bulkloader builder #6183 (and various other dependencies) go anywhere radical, or are those things being finalized at the same time?
    • ???

Changed

In part, I was wondering if the meeting was open.

@Jegelewicz
Copy link
Member Author

Thanks for documenting the implications

In part, I was wondering if the meeting was open.

All of our AWG meetings are "open".

@ewommack
Copy link

@lin-fred - just including you to double check I have the listserv info correct.

Just a quick note for this comment:

I think this probably should not be closed, and I think it should be communicated to the whole community rather than just the working group??

The email listserv goes to everyone now, so everyone who gets communication us through email should have gotten the email.

@lin-fred
Copy link
Contributor

Yes the AWG working group email is not just the AWG but everyone who wanted to sign up. If something like this needs to go out to more people, then maybe a newsletter write up, which then can be linked in a tweet/in the banner. I am unsure how else to reach more Arctos users.

@Jegelewicz
Copy link
Member Author

Jegelewicz commented Apr 27, 2023

Look at Geome model for building a template? We can't because of Bootstrap. But if we can get close it would be great. Adding definitions would be helpful.

@campmlc
Copy link

campmlc commented Apr 27, 2023

if Bootstrap were possible, we could solve a lot of difficult interface issues. I agree with trying to emulate Geome model.

@mkoo
Copy link
Member

mkoo commented Apr 27, 2023

FYI, I was using that as a model for functionality (not so much style but it's nice too)
So having definitions for fields and allowing flexible template building are key!

@dustymc
Copy link
Contributor

dustymc commented Apr 28, 2023

I think perhaps these last few comments refer to #6183??

Bootstrap doesn't make anything "possible" - it's just a tool, it makes some things easier, refuses to address others at all. I think maybe we've built up some misplaced mysticism or something??

Arctos field definitions are https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader (and almost none of that is going to survive this).

@Jegelewicz
Copy link
Member Author

Arctos field definitions are https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader (and almost none of that is going to survive this).

Assuming there will still be this thing? Working on this in Sheet 3 of https://docs.google.com/spreadsheets/d/1qstLM0xpW8gkkEnRxUpZWOZGtJkv2NTu8zIgKxv-nYc/edit?usp=sharing

@Jegelewicz
Copy link
Member Author

@dustymc In today's AWG - it was made clear that two things are missing from the columns listed in the file above

associated species
identification confidence - is this becoming identification rank? or are we also missing this and rank from the new list?

@dustymc
Copy link
Contributor

dustymc commented May 4, 2023

associated species

I think that got lost in #4707, I'll add it back in.

identification confidence

See #4829 (comment) via #5331 - the whole package should be considered.

becoming identification rank?

Is this a reference to #3540?

@Jegelewicz
Copy link
Member Author

Jegelewicz commented May 4, 2023

becoming identification rank?

Is this a reference to #3540?

Yes - how will that get added in data entry/bulkloading? IDENTIFICATION_ORDER?

@dustymc
Copy link
Contributor

dustymc commented May 4, 2023

identification_1_order | character varying(4) | | |

@Jegelewicz
Copy link
Member Author

Jegelewicz commented May 4, 2023

OK I think that covers it

associated_species being added back
identification confidence becoming an attribute of identification

Cool - do I need to fire up issues to create ctidentification_attributes? Then we will also need ctidentification_att_att so that we can link the controlled vocabularies for the new nature of id attribute to ctnature_of_id and the new identification confidence to ctidentification_confidence. So also issues for those terms....

@dustymc
Copy link
Contributor

dustymc commented May 4, 2023

o also issues

Yea, probably with the way things have been going. That's the code I'd written before one of the times this got killed, but I suppose I can just toss that.

If you're feeling brave can we PLEASE lose the "WE HAVE NOTHING TO SAY!!" option while we're in there?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement I think this would make Arctos even awesomer! Function-DataEntry/Bulkloading Priority - Wildfire Potential ignore this at everyone's peril, may smolder for now ...
Projects
None yet
Development

No branches or pull requests

6 participants