Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Editor] Improve a11y for newly added element (#15109) #15110

Merged
merged 1 commit into from
Jul 19, 2022

Conversation

calixteman
Copy link
Contributor

  • In the annotationEditorLayer, reorder the editors in the DOM according
    the position of the elements on the screen;
  • add an aria-owns attribute on the "nearest" element in the text layer
    which points to the added editor.

@calixteman calixteman linked an issue Jun 28, 2022 that may be closed by this pull request
@calixteman
Copy link
Contributor Author

@jcsteh, @MReschenberg I'd like to have your feedback.

@@ -479,6 +479,7 @@ class InkEditor extends AnnotationEditor {
#createCanvas() {
this.canvas = document.createElement("canvas");
this.canvas.className = "inkEditorCanvas";
this.canvas.setAttribute("aria-label", "User created image");
Copy link
Contributor

@timvandermeij timvandermeij Jul 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't all aria-labels in this patch fetch a localized string from the translation files?

@Snuffleupagus
Copy link
Collaborator

@calixteman Sorry about the delay in getting to this!
However, I was hoping for some feedback on the overall approach from the Firefox a11y-people, since you requested that in #15110 (comment), before trying to do an in-depth review of this PR.
(On the off chance that the implementation might change substantially, as a result of the a11y-feedback.)

Comment on lines 270 to 272
editor_ink_canvas_aria_label=User created image
editor_ink_aria_label=Ink Editor
editor_freetext_aria_label=FreeText Editor
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, both in naming and ordering, should this be formatted as follows instead?

Suggested change
editor_ink_canvas_aria_label=User created image
editor_ink_aria_label=Ink Editor
editor_freetext_aria_label=FreeText Editor
editor_free_text_aria_label=FreeText Editor
editor_ink_aria_label=Ink Editor
editor_ink_canvas_aria_label=User created image

Please also add the strings in https://github.com/mozilla/pdf.js/blob/master/web/l10n_utils.js as well, otherwise fetching of l10n-strings won't work correctly in general.

Comment on lines 106 to 122
const textLayer = this.div.parentNode
.getElementsByClassName("textLayer")
.item(0);
if (!textLayer) {
return shadow(this, "textLayerElements", null);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how most things in the viewer are asynchronous, the textLayer could potentially not yet exist when this code runs or at least not have been completely rendered yet; hence this could lead to intermittent behaviour/failures.

If the textLayer has finished rendering, its last element will be a div with a endOfContent-className. Otherwise you'd have probably have to, somehow, wait for the relevant "textlayerrendered"-event before running this code.

In any case, it seems to me that this code (and probably the rest of this patch) needs to be able to properly deal with the asynchronicity of the viewer here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some (possible) inspiration, please note how the structTree-functionality is handled in the viewer to account for the various kinds of asynchronicity involved:

pdf.js/web/pdf_page_view.js

Lines 785 to 815 in a1ac1a6

// The structure tree is currently only supported when the text layer is
// enabled and a canvas is used for rendering.
if (this.structTreeLayerFactory && this.textLayer && this.canvas) {
// The structure tree must be generated after the text layer for the
// aria-owns to work.
this._onTextLayerRendered = event => {
if (event.pageNumber !== this.id) {
return;
}
this.eventBus._off("textlayerrendered", this._onTextLayerRendered);
this._onTextLayerRendered = null;
if (!this.canvas) {
return; // The canvas was removed, prevent errors below.
}
this.pdfPage.getStructTree().then(tree => {
if (!tree) {
return;
}
if (!this.canvas) {
return; // The canvas was removed, prevent errors below.
}
const treeDom = this.structTreeLayer.render(tree);
treeDom.classList.add("structTree");
this.canvas.append(treeDom);
});
};
this.eventBus._on("textlayerrendered", this._onTextLayerRendered);
this.structTreeLayer =
this.structTreeLayerFactory.createStructTreeLayerBuilder(pdfPage);
}

Comment on lines 370 to 440
removePointerInTextLayer(editor) {
const { id } = editor;
const node = this.#textNodes.get(id);
if (!node) {
return;
}

this.#textNodes.delete(id);
let owns = node.getAttribute("aria-owns");
if (owns?.includes(id)) {
owns = owns
.split(" ")
.filter(x => x !== id)
.join(" ");
if (owns) {
node.setAttribute("aria-owns", owns);
} else {
node.removeAttribute("aria-owns");
}
}
}
Copy link
Collaborator

@Snuffleupagus Snuffleupagus Jul 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the textLayer has already been removed, as part of viewer clean-up (or just zooming), when this code runs; are we attempting to access dead DOM-elements in that case?

Could we perhaps avoid that sort of problem by re-factoring the #textNodes into a WeakMap instead?

src/display/editor/annotation_editor_layer.js Show resolved Hide resolved

FreeTextEditor._l10nPromise
.get("editor_freetext_aria_label")
.then(msg => this.div.setAttribute("aria-label", msg));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the asynchronicity of the code, would it be safer to do the following?

Suggested change
.then(msg => this.div.setAttribute("aria-label", msg));
.then(msg => this.div?.setAttribute("aria-label", msg));


FreeTextEditor._l10nPromise
.get("freetext_default_content")
.then(msg => this.editorDiv.setAttribute("default-content", msg));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the asynchronicity of the code, would it be safer to do the following?

Suggested change
.then(msg => this.editorDiv.setAttribute("default-content", msg));
.then(msg => this.editorDiv?.setAttribute("default-content", msg));


InkEditor._l10nPromise
.get("editor_ink_canvas_aria_label")
.then(msg => this.canvas.setAttribute("aria-label", msg));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the asynchronicity of the code, would it be safer to do the following?

Suggested change
.then(msg => this.canvas.setAttribute("aria-label", msg));
.then(msg => this.canvas?.setAttribute("aria-label", msg));


InkEditor._l10nPromise
.get("editor_ink_aria_label")
.then(msg => this.div.setAttribute("aria-label", msg));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the asynchronicity of the code, would it be safer to do the following?

Suggested change
.then(msg => this.div.setAttribute("aria-label", msg));
.then(msg => this.div?.setAttribute("aria-label", msg));

* @returns {number} Index of the first array element to pass the test,
* or |items.length| if no such element exists.
*/
function binarySearchFirstItem(items, condition, start = 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you expecting to use this code anywhere in the src/core/-folder?

If not, you probably want to place this in https://github.com/mozilla/pdf.js/blob/master/src/display/display_utils.js instead to avoid unnecessarily duplicating this code in both of the built pdf.js and pdf.worker.js files.

@calixteman
Copy link
Contributor Author

@calixteman Sorry about the delay in getting to this! However, I was hoping for some feedback on the overall approach from the Firefox a11y-people, since you requested that in #15110 (comment), before trying to do an in-depth review of this PR. (On the off chance that the implementation might change substantially, as a result of the a11y-feedback.)

I pinged Jamie and Morgan on slack, but today is the 4th of July, so I hope we'll have some feedback this week.

Copy link

@jcsteh jcsteh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Sorry for the delay in taking a look.

Is this expected to work on both tagged and untagged PDFs? It's hard for me to verify at present due to the aria-owns bug; see below.

const node = children[Math.max(0, index - 1)];
const owns = node.getAttribute("aria-owns");
if (!owns?.includes(id)) {
node.setAttribute("aria-owns", owns ? `${owns} ${id}` : id);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't working as expected. role="presentation" is incompatible with aria-owns because aria-owns requires that the element is in the a11y tree, but role="presentation" removes the element from the a11y tree. Firefox probably should ignore role="presentation" in this case (bug 1762116), but it currently doesn't. We can fix that in Firefox, but regardless, these attributes really shouldn't be set simultaneously. That is, role="presentation" should be removed if aria-owns is set and replaced when aria-owns is removed. Unfortunately, this is going to require changes to a bunch of other code here. :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I'll remove the presentation role and add it back.

@@ -213,7 +213,7 @@ class AnnotationEditor {
this.div.setAttribute("data-editor-rotation", (360 - this.rotation) % 360);
this.div.className = this.name;
this.div.setAttribute("id", this.id);
this.div.tabIndex = 100;
this.div.tabIndex = 0;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wrapping div shouldn't have a tabindex. The contenteditable is already focusable and we should rely on that for focusability. Otherwise, we end up with a pointless (arguably broken) tab stop here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contenteditable attribute is removed when the editor isn't in an editable state: for example when the user drags it to move it somewhere else on the page.
Anyway I'll remove the tab-index when the editor is in editing mode.
And there is too an "ink" editor in order to draw something on a pdf

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contenteditable attribute is removed when the editor isn't in an editable state: for example when the user drags it to move it somewhere else on the page.

Ah. It'll be hard for screen reader users to drag anyway, but regardless, I'm curious: what happens when a user drags the editor? Does it become non-editable altogether after that point or just while the user is holding down the mouse button?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A text editor can changed in double-clicking on it (simple click makes it the current one).
The idea behind being able to move is, is to help to adjust the position when you want to fill a form.


FreeTextEditor._l10nPromise
.get("editor_free_text_aria_label")
.then(msg => this.div?.setAttribute("aria-label", msg));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to focus, the role, aria-multiline and aria-label attributes must be set on the contentEditable itself (this.editorDiv), not the wrapping div.

@calixteman
Copy link
Contributor Author

Thanks for working on this. Sorry for the delay in taking a look.

Is this expected to work on both tagged and untagged PDFs? It's hard for me to verify at present due to the aria-owns bug; see below.

I had a look on a tagged pdf:
https://www.cdph.ca.gov/Programs/OHE/CDPH%20Document%20Library/Accessible-CDPH_OHE_Disparity_Report_Final%20(2).pdf

and I noticed the spans containing the text are inside some span with the class markedContent.
image

so I'm adding the aria-owns on the inner span (the one containing the text), is it correct or should it be on its parent ?

for (const span of el.querySelectorAll(
`span[role="presentation"]`
)) {
if (span.innerText.includes("adobe.com")) {

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization

'[adobe.com](1)' can be anywhere in the URL, and arbitrary hosts may come before or after it.
l10n/en-US/viewer.properties Outdated Show resolved Hide resolved
src/display/editor/annotation_editor_layer.js Outdated Show resolved Hide resolved
src/display/editor/annotation_editor_layer.js Outdated Show resolved Hide resolved
src/display/editor/annotation_editor_layer.js Outdated Show resolved Hide resolved
src/display/editor/annotation_editor_layer.js Outdated Show resolved Hide resolved
Copy link

@jcsteh jcsteh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is much better; thank you.

Is there supposed to be some way to dismiss the editor once you're done or does it just stay editable?

Also, how are text annotations rendered into the document after you save it? Any fixes there would probably be a separate piece of work, but I thought I'd ask here anyway.

@@ -213,7 +213,7 @@ class AnnotationEditor {
this.div.setAttribute("data-editor-rotation", (360 - this.rotation) % 360);
this.div.className = this.name;
this.div.setAttribute("id", this.id);
this.div.tabIndex = 100;
this.div.tabIndex = 0;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contenteditable attribute is removed when the editor isn't in an editable state: for example when the user drags it to move it somewhere else on the page.

Ah. It'll be hard for screen reader users to drag anyway, but regardless, I'm curious: what happens when a user drags the editor? Does it become non-editable altogether after that point or just while the user is holding down the mouse button?

@jcsteh
Copy link

jcsteh commented Jul 14, 2022

I had a look on a tagged pdf: https://www.cdph.ca.gov/Programs/OHE/CDPH%20Document%20Library/Accessible-CDPH_OHE_Disparity_Report_Final%20(2).pdf

and I noticed the spans containing the text are inside some span with the class markedContent. image

so I'm adding the aria-owns on the inner span (the one containing the text), is it correct or should it be on its parent ?

I don't think it matters. It seems to work as expected as is.

@calixteman calixteman force-pushed the editing_a11y branch 2 times, most recently from 9ce3678 to 4c0f442 Compare July 15, 2022 14:31
@calixteman
Copy link
Contributor Author

@Snuffleupagus , anything to add ?

@Snuffleupagus
Copy link
Collaborator

Sorry; I've not (yet) had time to look at this since the last round of changes, but will hopefully have time for that during the weekend.

Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this modifies the DOM of the textLayer, have you tested that highlighting of search results still work correctly with this patch (once new annotations are added)?

Naively, without having tested, it seems to me that this could potentially cause issues. E.g. consider the case where you first search and then add a new annotation, what happens to the existing search-highlight in that case?

Assuming that this does indeed work, would it be feasible to add an integration-test for searching in "edited" documents?

}

/**
* Update the toolbar if it's required to reflect the tool currently used.
* @param {number} mode
* @returns {undefined}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, and in number of places below, there's empty lines inside of JSDoc-comment blocks.
This is likely caused by search-and-replace, but please go through the patch and remove these unneeded new-lines.

return textChildren;
}

#hasTextLayer() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps make this a getter instead?


/**
* Find the text node which is the nearest and add an aria-owns attribute
* in order to position correctly this editor in the text flow.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar nit:

Suggested change
* in order to position correctly this editor in the text flow.
* in order to correctly position this editor in the text flow.

FreeTextEditor._l10nPromise.then(msg =>
this.editorDiv.setAttribute("default-content", msg)
);
this.editorDiv.id = `${this.id}-editor`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another case where I'm not sure that it's generally a good idea to add element ids to the DOM, since that makes them linkable.
Similar to other recent patches, can we use a custom data-element-id attribute instead?

(Also, if there's other pre-existing cases of ids being used in the Editor they probably need to be updated as well.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately aria-owns value must be an id or a list of ids:
https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Attributes/aria-owns#values
All the ideas are prefixed by:

const AnnotationEditorPrefix = "pdfjs_internal_editor_";

We can make it a bit more complex and dynamic to avoid any conflicts.

@@ -418,6 +429,10 @@ class InkEditor extends AnnotationEditor {
this.#fitToContent();

this.parent.addInkEditorIfNeeded(/* isCommitting = */ true);

// When commiting, the position of this editor is changed, hence we must
// move it at the right position in the DOM.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar nit:

Suggested change
// move it at the right position in the DOM.
// move it to the right position in the DOM.

Comment on lines 430 to 433
this.#eventBus._on(
"textlayerrendered",
this.#onTextLayerRendered.bind(this)
);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just like the event above, this one also needs to be removed on destruction.

@calixteman
Copy link
Contributor Author

Is there supposed to be some way to dismiss the editor once you're done or does it just stay editable?

Once the user finished to add some text, they can click outside the area of the editor to commit it.
A double-click on it will reopen it to make it editable.

Also, how are text annotations rendered into the document after you save it? Any fixes there would probably be a separate piece of work, but I thought I'd ask here anyway.

Text annotation are rendered as basic FreeText annotations it appears that I didn't think about a11y when saving: I need to check how it works in pdf format.
In HCM the color on screen is not necessarily the same as the one in the printed/saved pdf.

@calixteman
Copy link
Contributor Author

Given that this modifies the DOM of the textLayer, have you tested that highlighting of search results still work correctly with this patch (once new annotations are added)?

Naively, without having tested, it seems to me that this could potentially cause issues. E.g. consider the case where you first search and then add a new annotation, what happens to the existing search-highlight in that case?

The text layer is structurally not modified: only some aria-owns attributes are added on some nodes.
The node reordering is just applied in the annotationEditor layer and it has no impact on the search stuff.

Assuming that this does indeed work, would it be feasible to add an integration-test for searching in "edited" documents?

Yep I can but I'm not sure to see how it could be useful.
Do I miss something ?

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/467f21b0eddbec9/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Failed

Full output at http://54.241.84.105:8877/467f21b0eddbec9/output.txt

Total script time: 5.09 mins

  • Integration Tests: FAILED

@pdfjsbot
Copy link

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/57a129dacc30229/output.txt

Total script time: 12.04 mins

  • Integration Tests: FAILED

@calixteman
Copy link
Contributor Author

/botio integrationtest

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/75fab4705366967/output.txt

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 1

Live output at: http://54.193.163.58:8877/6a4f65955766292/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/75fab4705366967/output.txt

Total script time: 4.51 mins

  • Integration Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/6a4f65955766292/output.txt

Total script time: 10.03 mins

  • Integration Tests: FAILED

- In the annotationEditorLayer, reorder the editors in the DOM according
  the position of the elements on the screen;
- add an aria-owns attribute on the "nearest" element in the text layer
  which points to the added editor.
@calixteman
Copy link
Contributor Author

/botio integrationtest

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/18e5845e66ce512/output.txt

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/11ba4890119d0d6/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/18e5845e66ce512/output.txt

Total script time: 4.54 mins

  • Integration Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/11ba4890119d0d6/output.txt

Total script time: 7.16 mins

  • Integration Tests: FAILED

@Snuffleupagus
Copy link
Collaborator

/botio-windows integrationtest

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/8f5289254ea60f0/output.txt

@pdfjsbot
Copy link

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/8f5289254ea60f0/output.txt

Total script time: 9.66 mins

  • Integration Tests: FAILED

@Snuffleupagus
Copy link
Collaborator

/botio integrationtest

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/ad8ad86eccb330f/output.txt

@pdfjsbot
Copy link

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/d0929c543c7c034/output.txt

@pdfjsbot
Copy link

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/ad8ad86eccb330f/output.txt

Total script time: 4.55 mins

  • Integration Tests: Passed

@pdfjsbot
Copy link

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/d0929c543c7c034/output.txt

Total script time: 9.44 mins

  • Integration Tests: FAILED

@calixteman
Copy link
Contributor Author

@Snuffleupagus, once I've some time I'll try to fix the PageOpen issue on windows (I can't reproduce it locally).

@Snuffleupagus Snuffleupagus merged commit f46895d into mozilla:master Jul 19, 2022
@Snuffleupagus
Copy link
Collaborator

once I've some time I'll try to fix the PageOpen issue on windows (I can't reproduce it locally).

It's the same for me as well, it works just fine locally on Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make the newly added elements accessible
5 participants