attribute structure: do we need something else?

[dustymc]

All attributes include a free-text method. The idea that we need something more-structured comes up from time to time in various contexts.

I cannot support modifying method - we've tried that, it's too restrictive, I think if we do anything it must be an addition.

---

          I'm a bit worried about losing precision so I would advocate for an ear_length_methods code table that includes "from notch" and "from crown" as I'm certain a goodly amount of our volunteers and students will forget to record "from notch" as free text when entering records. There's still Remarks to accommodate pubs and other methods, plus the potential to expand the code table should we start measuring ears in novel ways, or need to add "other" and punt specifics to the Remarks field.

Originally posted by @ebraker in #6307 (comment)

[ebraker]

I totally get your angle here. I think relying on people to remember to include a free text string as a standard data entry protocol is not going to be effective, especially when moving away from our current controlled option. Maybe there could be an additional code table, 'method_standard' (or something)? Mammal measurement standards are a good candidate since they are well-established, and distinguish between different systems (Europe vs US for instance), so it would be useful to have a controlled vocabulary.

[dustymc]

it would be useful to have a controlled vocabulary.

I still don't think any replacement will scale. The method might involve detailed explanations of how hard the caliper was squeezed and the state of the animal and a brajillion other things that just can't be condensed down into a categorical code table.

MAYBE that free text should be supplemented with a category, and/or a formal link to publications, and/or - ????????????, but I don't think it can be REPLACED with any of that.

I stress "maybe" because I suspect that for most of these things most of the time the method just isn't recorded, and we're probably making the data worse by cramming it into some assumption-laden pigeonhole. I think we really need a strong use case to proceed with this, and I'm not sure I'm seeing it in the ear data (about a third of a percent of which claim to be from-crown).

[Jegelewicz]

I take this to mean we would add a field to all attributes as @ebraker mentioned - method_standard - which would be controlled by a code table and method would remain as is?

OR do some attributes get a method code table with any extra stuff (how I held the calipers) in remarks and some just allow free text in method?

[dustymc]

ake this to mean we would

It's an open question from my perspective - this sort of thing comes up from time to time, I've never quite been able to see the light, maybe someone has.

some attributes get a method code table

That's more complexity than I think anyone can deal with, I think this would have to be something that's available across the board.

in remarks

I think it would have to be some new thing, but ??

[dustymc]

@ebraker wants 'modifier'

attribute=gonad
modifier=left testis
value=....
units=...

[ebraker]

I was thinking it could be even more simplified...
attribute=gonad
position=right
value
units
remark (could insert 'ovary' or 'testis' here or this info is inferred from sex)

...but, RANGES may favor your example above (I just like the idea of creating a 'position' CT that could be used for a range of measurements)

[dustymc]

(Talking about localities, but attributes are attributes...)

@KatherineLAnderson @WaigePilson @Jegelewicz

would be useful if attribute values could reference data objects, such as (only?) Agents.

[dustymc]

Much of the problem with processing attributes into flat forms (see eg #6111) is that individual attributes may be encumbered, so I have to check sometimes dozens of times per record. I think that's simply not sustainable now, and it certainly can't scale to wherever #6179 might lead.

One approach might be to encumber types, which would require a change in the code table structure. (Eg, all https://arctos.database.museum/info/ctDocumentation.cfm?table=ctattribute_type#value are not public, making 'value' public would require another attribute with a different name.)

Maybe directly encumbering attributes (eg, a 'public BOOLEAN' option for each attribute) would work now, but I'm not sure that's scalable (nor usable) either.

[Jegelewicz]

directly encumbering attributes (eg, a 'public BOOLEAN' option for each attribute)

I feel like this is what people want, but nobody wants to manage the extra data? However, almost everything would default to "public" so the extra work would only be on those who want to encumber something? I believe this is how it works in many other systems (a checkbox to hide something). Creating duplicate (encumbered vs. not) seems like a worse option and more work to me.

[WaigePilson]

would be useful if attribute values could reference data objects, such as (only?) Agents.

Yes, I would love to be able to reference a specific Agent as e.g., the landholder of a Locality. Maybe I'm missing something, but isn't this already possible for some attributes that you are marking the determiner for? Is the suggestion something different here?

I think I'm lost on the rest of this thread, so I'll just comment on the original tagged question!

[dustymc]

lost on the rest of this thread

What should Attributes do? (And maybe some how, and sorta-related, and whatever, but mostly I want to know what people want to do.)

determiner

Yes, all attributes have a determiner, and for some the documentation suggests using it for things that aren't quite "determiner." (And maybe that's enough, there is definitely value in simplicity.) I think I'm hearing that we need a second THING with identical structure:

• attribute type: whatever
• attribute method: something
• attribute value (or some new thing, or ???): https://arctos.database.museum/agent/21300608
• attribute determiner: https://arctos.database.museum/agent/21346927

and following that, would that "value" (or new beside-value-thing or whatever) need reference only agents, or also other data objects eg

• attribute type: whatever
• attribute method: something
• attribute value (or some new thing, or ???): https://arctos.database.museum/publication/10011907
• attribute determiner: https://arctos.database.museum/agent/21346927

[Jegelewicz]

The ability to use attribute value: https://arctos.database.museum/agent/21300608 would be a nice start. Then I could say things like

attribute type	attribute value	attribute determiner	attribute determined date	attribute method	attribute remark
landholder	Mesa Verde National Park	Teresa J. Mayfield-Meyer	2023-08-31	review of the accession documents

instead of

attribute type	attribute value	attribute determiner	attribute determined date	attribute method	attribute remark
landholder	US National Park Service	Mesa Verde National Park	2023-08-31	review of the accession documents by Teresa Mayfield-Meyer
landholder	US National Park Service	Teresa J. Mayfield-Meyer	2023-08-31	review of the accession documents	Mesa Verde National Park

[dustymc]

From OGL discussions: "standard method" (pick) would be useful

attribute type=DNA thingee
standard method={pick from list - some kit maybe}
method=finicky details...

Not from them, this also seems a place where adding a publication reference (eg for 'how to use the kit, according to...') would be handy.

[dustymc]

Hinted at (maybe accidentally) in Issues meeting:

• attribute_value
• internal_attribute_value

?!

[Jegelewicz]

From today's discussion - code table terms are data and as such could/should hold their own metadata. This might provide us places to stash Github issue links, alternate term names (synonyms), tags (i.e. repro or skeletal), identifiers that link the term to external sources from which a definition was borrowed, the collections using the term - and probably other things I am missing.

Possible?

[dustymc]

Possible

Yes, and anything that can be unloaded from the "data" to the "metadata" should make kinda-everything easier to deal with (less data, less complexity, easier to flatten, etc.).

[dustymc]

https://github.com/ArctosDB/internal/issues/303#issuecomment-1972145286 is a possible use case for a public/private flag on the type.

[campmlc]

From issues meeting 2024-09-05
Request to have possibility of controlled vocab table for positions (left, right, left and right, unknown) in methods to deal with gonad and other length and width. Possible to allow certain attributes to have controlled vocab methods dropdowns linked to a specific code table, while others are free text?
@bryansmclean @happiah-madson @ewommack @mkoo

[dustymc]

Some comments related to the 2024-08-05 AWG meeting https://docs.google.com/document/d/1WvDYixTFZU9ZIFREwF5t605cywxkkQWOW42FfgOV8I8/edit#heading=h.q3pk226foene

Here are some ways we might accommodate more-detailed data.

1. Lots of attributes.
   • good: use it, nothing to do
   • not so great: it's easy to get lost in lots of THINGs
     • yea but: lots of recent discussions lean towards the idea that people can deal with lots of THINGS better than they can deal with fewer complex THINGS.
2. fewer more general attributes, supplemented by some sort of more complexity (for gonad that might involve [left, right, ovary, teste, etc.]) in the structure
   • good: fewer first-level THINGs to get lost in
   • not so great: users near-universally complain about more complexity, and this would add more - maybe a lot more (eg would this result in multidimensional code tables??) - complexity
   • not so great: there's no pattern which I can see emerging, every category of attribute seems to need some different addition/expansion (see above re: use restrictions, agents, etc. - I don't believe this is as "simple' as some "secondary categorical data"), this probably requires some sort of modeling exercise
   • not so great: this would likely require extensive development, above and beyond solidifying some model
3. fewer more general attributes, make more use of free-text method
   • good: use it, nothing to do
   • good: super-simple entry, just type whatever's on the datasheet
   • not so great: data are likely lossy, difficult to extract
     • yea but: Could AI help? There are definitely patterns in these data, even if they're difficult for humans to parse, and there is definitely value in having all similar data in fewer (one is a nice number!) places.
4. fewer very generalized (anywhere from "data" to "mammal repro data according to {schema}")JSON attributes
   • good: use it, nothing to do
   • good: any amount of complexity or detail is no obstacle. Need 52 "fields" to hold all the information about each one of the 37 ear measurements? No problem!
   • good/not-so-great: little or no need to involve The Community in how you record data; just develop a schema (something formal, a spreadsheet template, whatever) and use it. (Likely not a problem if some community develops, but possibly not useful data if everyone does their own thing)
   • good/not-so-great: any Arctos UI would need developed if it becomes necessary, but eg Excel entry should be trivial
   • good/not-so-great: displaying data through Reports should be trivial (eg assuming it can be munged into a CSV-friendly format), but these data would not be expected to fit well in tabular displays mixed with other data (eg search results)
   • good: this would be easily extendable to any CMS that can store text.
5. Do (4), but as a webservice that anyone can use. (Why again are we not just using https://futres.org/?? Arctos has an API and good identifiers, should be trivial!)

---

I like (1) because of its simplicity and accessibility (but expect some users would get lost in the multitude of options).

(2) could be pretty awesome but probably needs significant time, development, and funding (eg that would ideally start with an extensive round of interviews leading to a functional requirements document), and would near-inevitably make Arctos more complicated.

I don't like (3) very much, but do wonder if it's not sufficient to support the actual research (and that's the point of all of this, no?). I suspect the entry folks would love it, as would the people training them. I also suspect this is the one thing that might be used somewhat homogeneously across collections, and so I expect this would be more discoverable than the alternatives.

I like (4) because it's near-infinitely scalable - WHATEVER the next new Arctos collection wants to record in ridiculous detail, this should support that.

I have no idea why we've not all been using (5) for years. (Because Arctos is the only CMS which has identifiers and APIs to support such things??)