get error logs to the people who need to see them

[dustymc]

Is your feature request related to a problem? Please describe.

Errors are not sorted. Anyone can sign up to receive error logs (global settings), but then they'll get all logs, get overwhelmed, and not see relevant things.

Describe what you're trying to accomplish

I try to forward potentially problematic errors to appropriate users, but (1) certainly miss things, and (2) I send them things they don't need to see. I can't tell if someone's doing something dangerous, or someone who knows what they're doing has made a mistake - it all looks scary from here!

Describe the solution you'd like

Somehow filter logs to appropriate collection personnel. Maybe a new collection contact role?

Describe alternatives you've considered

Existing system

Additional context

This would make it much easier for "supervisors" (eg, collection managers) to see what their techs/students/etc. are doing. Hopefully this would lead to a better awareness of what's going on in the collection, but also to things like Arctos change requests ("we keep seeing that error because the form is weird") and new tutorials.

[Jegelewicz]

Maybe a new collection contact role?

Maybe, but why not just those with "manage collection"?

However, does this cover shared stuff like media, localities and taxonomy? Maybe those error logs should go to the respective committee?

[dustymc]

why not just those with "manage collection"?

Suits me, and should be easy enough to fine-tune if necessary - using some sort of collection contact would be most of the dev work.

Shared stuff like media, localities and taxonomy? Maybe those error logs should go to the respective committee?

Excellent idea. I think that would just be adding some "fields" to global settings (and using them to sort).

[dustymc]

#1955 logs username in tables and should make this much more approachable.

It's also addressed the "overwhelming" aspect of signing up for logs - rather than hundreds of emails per hour, Arctos is currently sending out one summary per hour. The schedule might change, but it should remain a much more manageable load.

[Jegelewicz]

@mkoo suggests a subscription format - get the notifications that you really want.

[dustymc]

Can we somehow do this through agents (and related agents, er sumthin') rather than by adding some dedicated mail-this-to-them structure? Eg all taxonomy issues go to one agent (and their friends), all msb-access emails go to msb institutional agent, etc.

[Jegelewicz]

@lin-fred this can be something for agent committee to work on - see above.

[dustymc]

Will be current relationships - group agents need transmogrified

[Jegelewicz]

From Agenda

Lower the email load - internal issue #116
These are the notifications that you get when something gets a notice from Arctos -> when there is a force a loan, force a taxon name, problem with an agent name, etc.
Can we have a list of things we can subscribe to for Arctos notices, maybe something that could go into the profile in Arctos.
Will take structural changes -> so things have to change in how Arctos is structured for email list groups
Can we create more targeted listservs that the different emails can go?
Before design something fancy like the selection of what emails you get, here we manage listservs to help Dusty winnow down who gets the different warning emails.
But how will we design the listservs? The different chairs of the committees?
Can we pull it through Arctos Agent groups? That way not another outside organization system.
Something like this: https://arctos.database.museum/agents.cfm?agent_id=21318826 (remember to log in)
And https://arctos.database.museum/agents.cfm?agent_id=21326709
#2550 (comment)

[Jegelewicz]

Arctos Working Group Officers are set up, but the agent includes not only the group email but the individual officer emails - not sure if that will mean two emails for everyone in the list?

[dustymc]

There's now a function for this: https://github.com/ArctosDB/PG_DDL/blob/master/function/getRelatedAgentEmail.sql

Note the comment:

Some of the involved agents do not have logins, so this DOES NOT follow
the general rules of only sending emails to people with active accounts;
this will send to agents which are not people and do not have accounts at all.

I'm not crazy about that, but I don't know how to avoid it either.

Demo (from production):

arctosprod@arctos>>  select getRelatedAgentEmail(agent_id) from agent where preferred_agent_name='Arctos Working Group Officers';
                                                                                                                                         getrelatedagentemail                                                                                                                                          
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 andrew.doll@dmns.org,arctos.chair@gmail.com,arctos.communications@gmail.com,arctos.membership@gmail.com,arctos.treasurer@gmail.com,arctos-working-group-officers@googlegroups.com,campmlc@unm.edu,emily.braker@colorado.edu,Emily.Braker@colorado.edu,lindsey.frederick@state.nm.us,mkoo@berkeley.edu
(1 row)

"Global" implementation would be pretty straightforward - add

cf_global_settings.taxonomy_problem_email_agent_id int references agent(agent_id)

and for "global taxonomy problems" just

select getRelatedAgentEmail(cf_global_settings.taxonomy_problem_email_agent_id)

Whoever's controlling those emails would be responsible for making sure they can receive. Suggest we add "send from everything before _email_agent_id" (or whatever it turns out to be - taxonomy_problem@arctos.database.museum with the above example) to https://handbook.arctosdb.org/how_to/developer-guide.html

Assuming this all sounds reasonable, I'd just need a list of new keys to make that happen. It's probably worth moving the existing email lists to the agent model as well - those are

        Column         |            Type             | Collation | Nullable | Default 
-----------------------+-----------------------------+-----------+----------+---------
 bug_report_email      | character varying(4000)     |           |          | 
 data_report_email     | character varying(4000)     |           |          | 
 log_email             | character varying(4000)     |           |          | 

but I don't think they're used very consistently - maybe replace all of those with a default "general_problem_email_agent_id" or something??

---

Part Two: That doesn't address collection-specific email routing.

I think we first need a new dedicated value in https://arctos.database.museum/info/ctDocumentation.cfm?table=ctcoll_contact_role - "log receiver" or something of the sort.

I think I'd like to somehow allow agents like https://arctos.database.museum/agents.cfm?agent_id=21334678 for that, but that would require sending more email to not-users, which I'm never going to be crazy about.

I think I DO NOT want to follow relationships for that - StudentA (probably) doesn't need to see StudentZ's logs, even if they work in the same collection. If StudentA does need to see collection logs for some reason, they can just be directly added as a collection contact of the appropriate type.

From there, any error that involves an operator would go to operator's collection(s)--->collection "log receiver" contact(s); "Museum of Vertebrate Zoology Bird Collection" (among others) would get emails (assuming they have a valid email address) when a cat wanders across Carla's keyboard.

Workable?

[dustymc]

NEEDED: #3768 and #4013 (and probably others) need a "data quality people with dba-type skills" contact - hard-coding them for now

[Jegelewicz]

@dustymc does the TURD cover this?

[Jegelewicz]

Closing due to lack of response - I feel like this has now been covered by the TURD.

[dustymc]

covered by the TURD.

That's completely unrelated.

[Jegelewicz]

@dustymc what is it that we need to do here? How are notifications involved? I am convinced that sending emails is not going to change anything. I do think we could ask some committees to review logs at monthly meetings, but we cannot rely on ANYONE to respond immediately to any error log emails. I cannot help, because I am not clear on what is needed from the community.

[dustymc]

we need to do

Probably nothing, it's just not floated to the top of my pile.

How are notifications involved?

It's a mechanism to communicate with users (hopefully) without melting infrastructure.

cannot rely on ANYONE to respond immediately to any error log emails.

Nobody's asking for that, this would just make it easier to see when someone in your collection is having problems. Is that not clear from the initial comment?

[Jegelewicz]

Can we close this?

[mkoo]

Closed