feat(FieldFormatters): add visual editor #5075

maxpatiiuk · 2024-07-07T00:30:23Z

Fixing an issue from 2012 😊

Fixes #23

Checklist

Self-review the PR after opening it to make sure the changes look good
and self-explanatory (or properly documented)
Add automated tests
Add relevant issue to release milestone

Testing instructions

Generally visual editor for Field Formatters should act and look very similar to the one for Record formatters since it shares a lot of code with it.

To test:

Verify that each field formatter type (constant, any character, ...) can be set in the UI, without user-experience issues, and after saving, can be read back correctly
Verify that a formatter created/edited using a visual editor works in Specify 7 and Specify 6 (in forms and query results)
- For reference, here is how to configure a field formatter in Specify 6
P.S: don't try to enter an emoji character like on the screenshot above - Specify 7 back-end does not support emojis in app resources 😢

Documentation on field formatters

maxpatiiuk · 2024-07-07T00:32:39Z

specifyweb/frontend/js_src/lib/components/FieldFormatters/FieldFormatter.tsx

+            if (mapping !== undefined && mapping?.length > 1)
+              softError('Expected mapping length to be no more than 1');
+            const field = mapping?.[0];
+            if (field?.isRelationship === true) {
+              softError(
+                'Did not expect relationship field in field formatter mapping'
+              );


these cases should not happen - the UI doesn't allow them. but handling them here just in case (and to satisfy TypeScript)

specifyweb/frontend/js_src/lib/components/SchemaConfig/Format.tsx

grantfitzsimmons · 2024-07-08T13:12:44Z

❤️

grantfitzsimmons

When creating a field format like GiftNumber using the regular expression /^(KUI|KUBI|NHM)$/, I am unable to set it up correctly.

See that I am supposed to put the 'hint' on the left and 'pattern' on the right:

This works as expected when using a regex validator, but when setting it as the pattern it is not recognized properly.

	<format system="true" name="GiftNumber" class="edu.ku.brc.specify.datamodel.Gift" fieldname="giftNumber" title="" default="true">
		<autonumber>edu.ku.brc.af.core.db.AutoNumberGeneric</autonumber>
		<field type="regex" size="4" value="/^(KUI|KUBI|NHM)$/" byyear="true" pattern="KUBI"/>
	</format>

After modifying other sections of the format, those changes are not made immediately visible for the 'Example Field'. Instead, it uses the previous field format before the edits have been made.

See that it shows "Required Format: KUBI" in the tooltip:

(This tooltip is most often obscured by the browser's "match the requested formatter" error)
https://github.com/specify/specify7/assets/37256050/d34bcb16-85d5-43fc-8c68-56bef2e36cc6

After making it simply numeric:

What does the 'Auto Numbering' do that the 'Auto-number' checkbox does not?

maxpatiiuk · 2024-07-13T15:07:28Z

When creating a field format like GiftNumber using the regular expression /^(KUI|KUBI|NHM)$/, I am unable to set it up correctly.

Looks like the expected regex between Sp6 and Sp7 differed.
In Specify 6, the regex is run on each field part separately, so /^ and $/ are necessary
In Specify 7, all regexes are joined together into on regex, thus, having /^ or $/ will break things if there is more than one part in your formatter.

Fixed:

When reading from .xml, both Specify 7 front-end and back-end will trim any /^ or $/ from the regex pattern to not break the pattern when multiple parts are joined together. Similarly, outer ( and ) are removed if present
When front-end saves the updated .xml, the syncers will make sure to add /^ and $/, even if not originally specified by the user. Also, if the pattern includes |, leading ( and trailing ) will be added

In practice, that means:

When using the visual field format editor, it doesn't matter anymore if you put or not put /^ and $/ in a regex field as the value will be normalized automatically
Backwards compatibility with Specify 6

maxpatiiuk · 2024-07-13T15:11:49Z

See that I am supposed to put the 'hint' on the left and 'pattern' on the right:

yeah, it's not ideal that 'hint' comes before the 'pattern' as the opposite order seems more intuitive.

The reason I did it this way is because of the column heading.
For all other part types, the 'hint' column is used both for declaring what the format should be, and for proving the validation message
For regex parts, since regex is not "user friendly" enough to put into a regex message, the 'hint' field is used only for validation messages, and the regex pattern itself is a separate field (which also validates that the regex pattern has correct syntax)

Open to ideas on how to make this more intuitive

maxpatiiuk · 2024-07-13T15:28:55Z

After modifying other sections of the format, those changes are not made immediately visible for the 'Example Field'. Instead, it uses the previous field format before the edits have been made.

Fixed

maxpatiiuk · 2024-07-13T15:33:42Z

What does the 'Auto Numbering' do that the 'Auto-number' checkbox does not?

Ups, the "Auto Numbering" checkbox at the top is not supposed to be exposed in the UI - removed
(it corresponds to whether .xml had a tag like <autonumber>edu.ku.brc.specify.dbsupport.CollectionAutoNumber</autonumber>, which is an sp6-only construct - sp7 cares about it only in so far as creating sp6-compatiable field formatters)

Fixes #5075 (review)

grantfitzsimmons · 2024-07-19T17:00:20Z

@maxpatiiuk Can you resolve the merge conflicts?

lexiclevenger

Testing instructions

Generally visual editor for Field Formatters should act and look very similar to the one for Record formatters since it shares a lot of code with it.

To test:

Verify that each field formatter type (constant, any character, ...) can be set in the UI, without user-experience issues, and after saving, can be read back correctly
Verify that a formatter created/edited using a visual editor works in Specify 7 and Specify 6 (in forms and query results)
- For reference, here is how to configure a field formatter in Specify 6
P.S: don't try to enter an emoji character like on the screenshot above - Specify 7 back-end does not support emojis in app resources 😢

Super exciting! I have a few recommendations:

If no default formatter is selected, the message says "record formatter. I would change this to "field formatter"

The default for the "size" field for most field types is 0, and this can be saved as long as there is a value in the "hint" field. The value will be shown in the formatter, but it will make the format invalid if that value is entered in the field. In the Specify 6 field formatter, the "size" value cannot go below 1; I think adding this in 7 would be a good solution.

The user can input and save any value at any length in the "hint" field (see screenshot above) for the numeric field type, but this is misleading because "#" is the only character that actually works, and it will only appear as many times as the value in the "size" field. Maybe this field could be read-only since it's not really functional.

I'd like to hear what others have to say about 2 and 3; they don't conflict with the xml editor, and the "Example field" displays their consequences correctly, but I think these solutions would make the visual editor more user-friendly.

maxpatiiuk · 2024-07-21T18:46:32Z

If no default formatter is selected, the message says "record formatter. I would change this to "field formatter"

Good point. To have the same message be applicable both in the field formatters editor and record formatters editor, I edited it to say:

"Please designate one of the formatters as default"

The default for the "size" field for most field types is 0, and this can be saved as long as there is a value in the "hint" field. The value will be shown in the formatter, but it will make the format invalid if that value is entered in the field. In the Specify 6 field formatter, the "size" value cannot go below 1; I think adding this in 7 would be a good solution.

Made "size" value not go below 1

The user can input and save any value at any length in the "hint" field (see screenshot above) for the numeric field type, but this is misleading because "#" is the only character that actually works, and it will only appear as many times as the value in the "size" field. Maybe this field could be read-only since it's not really functional.

Fixed. "hint" is now readonly for number fields

Fixes #5075 (review)

lexiclevenger

Looks good! Another thing I noticed is that clicking on the "edit" button next to a format in schema config often takes you to the wrong one.

Schema.Config_.Gift._.Specify.7.-.Google.Chrome.2024-07-23.11-30-30.mp4

https://fwri1924-field-editor.test.specifysystems.org/specify/schema-config/en/Gift/

- Field is ambiguous name - Field suggests that you can have fields inside of a formatter - you can't - you can only have text parts inside a field formatter

Triggered by 06b3c1a on branch refs/heads/field-editor

Fixes #5075 (review)

Triggered by b4736f8 on branch refs/heads/field-editor

Fixes #5075 (review)

maxpatiiuk · 2024-07-30T00:29:10Z

Fixed!
For testing, to reproduce the bug look at formatters for tables that are not first in the alphabetical order (so all except for Accession 😊)

lexiclevenger

The edit button takes you to the correct resource now! One more issue I didn't notice before (sorry!) is that the save button is enabled as soon as the "Field Formatters" dialog is opened even if no changes have been made.

Screen.Recording.2024-08-05.at.2.47.07.PM.mov

maxpatiiuk added this to the 7.9.7 milestone Jul 7, 2024

maxpatiiuk self-assigned this Jul 7, 2024

maxpatiiuk commented Jul 7, 2024

View reviewed changes

maxpatiiuk requested review from grantfitzsimmons, acwhite211 and a team July 7, 2024 00:56

maxpatiiuk removed this from the 7.9.7 milestone Jul 7, 2024

realVinayak reviewed Jul 7, 2024

View reviewed changes

specifyweb/frontend/js_src/lib/components/SchemaConfig/Format.tsx Outdated Show resolved Hide resolved

realVinayak requested a review from a team July 7, 2024 02:28

grantfitzsimmons requested changes Jul 8, 2024

View reviewed changes

maxpatiiuk requested review from a team and removed request for acwhite211 July 13, 2024 15:45

maxpatiiuk added a commit that referenced this pull request Jul 13, 2024

fix(FieldFormatters): normalize ^ and $ in regex

d8ef2a5

Fixes #5075 (review)

maxpatiiuk force-pushed the field-editor branch from f5b2b65 to b4736f8 Compare July 13, 2024 15:46

maxpatiiuk requested review from grantfitzsimmons and a team July 13, 2024 15:46

lexiclevenger requested changes Jul 19, 2024

View reviewed changes

lexiclevenger requested a review from a team July 19, 2024 19:42

maxpatiiuk added a commit that referenced this pull request Jul 21, 2024

fix(FieldFormatters): normalize ^ and $ in regex

9b664c9

Fixes #5075 (review)

maxpatiiuk added a commit that referenced this pull request Jul 21, 2024

fix(FieldFormatters): resolve minor UX issues

22ced94

Fixes #5075 (review)

maxpatiiuk force-pushed the field-editor branch from 91bf354 to 22ced94 Compare July 21, 2024 18:47

maxpatiiuk requested a review from lexiclevenger July 21, 2024 18:47

lexiclevenger requested changes Jul 23, 2024

View reviewed changes

maxpatiiuk added 15 commits July 29, 2024 17:03

feat(FieldFormatter): begin implementing visual editor

041d3e9

refactor(FieldFormatters): cleanup placeholder & regex semantics

e814d89

refactor(FieldFormatter): rename "Field" to "Part"

72ccd1d

- Field is ambiguous name - Field suggests that you can have fields inside of a formatter - you can't - you can only have text parts inside a field formatter

feat(FieldFormatters): complete visual editor

5a91ccd

feat(FieldFormatters): improve visual editor UX

9be3eb4

tests(FieldFormatters): update failing tests

174e411

Lint code with ESLint and Prettier

a3771b0

Triggered by 06b3c1a on branch refs/heads/field-editor

refactor(FieldFormatters): do a self code review

977ba5a

feat(SchemaConfig): add link to FieldFormatter VisualEditor

e33b832

fix(FieldFormatters): normalize ^ and $ in regex

765786b

Fixes #5075 (review)

fix(FieldFormatters): correctly resolve formatter in editor preview

494d294

fix(FieldFormatters): don't expose internal autoNumbering state in UI

1505ffd

Lint code with ESLint and Prettier

8fea06e

Triggered by b4736f8 on branch refs/heads/field-editor

fix(FieldFormatters): resolve minor UX issues

8e70d3d

Fixes #5075 (review)

fix(FieldFormatters): fix schema config indexes being off

77b5152

Fixes #5075 (review)

maxpatiiuk force-pushed the field-editor branch from 22ced94 to 77b5152 Compare July 30, 2024 00:28

maxpatiiuk requested a review from lexiclevenger July 30, 2024 00:29

lexiclevenger requested changes Aug 5, 2024

View reviewed changes

Merge branch 'production' into field-editor

0634506

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(FieldFormatters): add visual editor #5075

feat(FieldFormatters): add visual editor #5075

maxpatiiuk commented Jul 7, 2024 •

edited

Loading

maxpatiiuk Jul 7, 2024

grantfitzsimmons commented Jul 8, 2024

grantfitzsimmons left a comment

maxpatiiuk commented Jul 13, 2024 •

edited

Loading

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

grantfitzsimmons commented Jul 19, 2024

lexiclevenger left a comment

maxpatiiuk commented Jul 21, 2024 •

edited

Loading

lexiclevenger left a comment

maxpatiiuk commented Jul 30, 2024

lexiclevenger left a comment

feat(FieldFormatters): add visual editor #5075

Are you sure you want to change the base?

feat(FieldFormatters): add visual editor #5075

Conversation

maxpatiiuk commented Jul 7, 2024 • edited Loading

Checklist

Testing instructions

maxpatiiuk Jul 7, 2024

Choose a reason for hiding this comment

grantfitzsimmons commented Jul 8, 2024

grantfitzsimmons left a comment

Choose a reason for hiding this comment

maxpatiiuk commented Jul 13, 2024 • edited Loading

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

grantfitzsimmons commented Jul 19, 2024

lexiclevenger left a comment

Choose a reason for hiding this comment

Testing instructions

maxpatiiuk commented Jul 21, 2024 • edited Loading

lexiclevenger left a comment

Choose a reason for hiding this comment

maxpatiiuk commented Jul 30, 2024

lexiclevenger left a comment

Choose a reason for hiding this comment

maxpatiiuk commented Jul 7, 2024 •

edited

Loading

maxpatiiuk commented Jul 13, 2024 •

edited

Loading

maxpatiiuk commented Jul 21, 2024 •

edited

Loading