[api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) #18134

calixteman · 2024-05-21T13:28:49Z

Right now, editable annotations are using their own canvas when they're drawn, but it induces several issues:

if the annotation has to be composed with the page then the canvas must be correctly composed with its parent. That means we should move the canvas under canvasWrapper and we should extract composing info from the drawing instructions... Currently it's the case with highlight annotations.
we use some extra memory for those canvas even if the user will never edit them, which the case for example when opening a pdf in Fenix.

So with this patch, all the editable annotations are drawn on the canvas. When the user switches to editing mode, then the pages with some editable annotations are redrawn but without them: they'll be replaced by their counterpart in the annotation editor layer.

calixteman · 2024-05-21T13:29:06Z

/botio test

moz-tools-bot · 2024-05-21T13:29:10Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/848b5ec2a6372b9/output.txt

moz-tools-bot · 2024-05-21T13:29:10Z

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/faccdf624756a1a/output.txt

moz-tools-bot · 2024-05-21T13:57:29Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/848b5ec2a6372b9/output.txt

Total script time: 28.31 mins

Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 38
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/848b5ec2a6372b9/reftest-analyzer.html#web=eq.log

moz-tools-bot · 2024-05-21T14:12:25Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/faccdf624756a1a/output.txt

Total script time: 43.23 mins

Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 11

Image differences available at: http://54.193.163.58:8877/faccdf624756a1a/reftest-analyzer.html#web=eq.log

calixteman · 2024-05-21T14:27:14Z

/botio test

moz-tools-bot · 2024-05-21T14:27:18Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 1

Live output at: http://54.241.84.105:8877/f5671dedc2e3b62/output.txt

moz-tools-bot · 2024-05-21T14:27:18Z

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 1

Live output at: http://54.193.163.58:8877/e5817c6d4a755b2/output.txt

Snuffleupagus

I'll try to review this properly later (probably next week since it's a lot of code), however I unfortunately have several reservations regarding the API so setting r- for now.

moz-tools-bot · 2024-05-21T14:57:55Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/f5671dedc2e3b62/output.txt

Total script time: 28.63 mins

Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 36
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/f5671dedc2e3b62/reftest-analyzer.html#web=eq.log

moz-tools-bot · 2024-05-21T15:20:36Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/e5817c6d4a755b2/output.txt

Total script time: 43.12 mins

Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 16

Image differences available at: http://54.193.163.58:8877/e5817c6d4a755b2/reftest-analyzer.html#web=eq.log

timvandermeij · 2024-05-21T16:49:34Z

@calixteman Given the size of the patch would it be an idea, to make reviewing easier by reducing the number of changes in this PR, to extract the test refactorings into a separate PR? I'd be happy to review that part and we can then land it first so @Snuffleupagus can check the remainder of the PR later. Those test refactorings (introduction of all switch<mode> helper functions and the two test utility functions) are useful to have in general and make all integration tests consistent which would also help for other integration test PRs such as the waitForTimeout and the Chrome BiDi ones (the more improvements we have in place, the better).

calixteman · 2024-05-29T12:45:13Z

@calixteman Given the size of the patch would it be an idea, to make reviewing easier by reducing the number of changes in this PR, to extract the test refactorings into a separate PR? I'd be happy to review that part and we can then land it first so @Snuffleupagus can check the remainder of the PR later. Those test refactorings (introduction of all switch<mode> helper functions and the two test utility functions) are useful to have in general and make all integration tests consistent which would also help for other integration test PRs such as the waitForTimeout and the Chrome BiDi ones (the more improvements we have in place, the better).

Yep, it has been done in #18165.

Patch updated

Snuffleupagus

Sorry to say, but I'm finding this somewhat difficult to wrap my head around w.r.t. both the API and viewer (which is why I've not done a proper review yet). Any chance that it could be somehow split into smaller parts and/or simplified somehow?

Given that re-rendering of a page can in some cases be slow, how does this perform when switching to/from editing-mode?

Also, I really don't at all understand how the API works since it seems that you're "mixing" page-specific and global state on the worker-thread; i.e. the EndEditingSession functionality.
Can't this lead to intermittent issues if that changes while a page is parsing/rendering?
To me it seems that the editing-state needs to be constant for a getOperatorList invocation.

There's also the way that the intentState is now handled in the API, adding complexity in the render-method, which suggests to me that the cacheKey isn't correctly computed in the getRenderingIntent-helper.

calixteman · 2024-05-31T17:31:41Z

Given that re-rendering of a page can in some cases be slow, how does this perform when switching to/from editing-mode?

If a page with an editable annotation is slow to render then switching to editing mode will be slower.
It behaves like this in Acrobat and I don't really see how we can do better.
The main advantages of this approach are:

less memory consumption;
better rendering of highlights (when it'll be possible to edit them).

That said if you've any good ideas, as usual please share.

marco-c · 2024-06-07T09:23:07Z

This is the first step for bug 1883884.

src/display/api.js

calixteman · 2024-06-27T15:57:08Z

/botio integrationtest

moz-tools-bot · 2024-06-27T15:57:10Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/9cab3b8d503873c/output.txt

moz-tools-bot · 2024-06-27T15:57:10Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/c13dc56b765177d/output.txt

moz-tools-bot · 2024-06-27T16:04:57Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/9cab3b8d503873c/output.txt

Total script time: 7.77 mins

Integration Tests: Passed

moz-tools-bot · 2024-06-27T16:15:35Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/c13dc56b765177d/output.txt

Total script time: 18.41 mins

Integration Tests: FAILED

Snuffleupagus

Edit: Given that we have a pending release (hopefully soon), we should probably not land this PR just prior to that.

I've not tested this very much, so I'm counting on you having tested this thoroughly during development :-)
Also, the code feels a lot more approachable now compared to the initial version; thanks for that!

r=me, with a couple of final comments addressed; thank you!

test/integration/test_utils.mjs

test/integration/annotation_spec.mjs

web/pdf_viewer.js

timvandermeij · 2024-07-01T18:13:15Z

Moreover, since this will probably land as the first [api-minor] change after the new release, please also bump the library version in a separate PR after merging this (so there is a merge commit) similar to e.g. 648a2d8.

calixteman · 2024-07-01T19:43:55Z

/botio integrationtest

moz-tools-bot · 2024-07-01T19:43:57Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/c5e4786d5ec3077/output.txt

moz-tools-bot · 2024-07-01T19:43:57Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/77b2cda57cb3a6a/output.txt

moz-tools-bot · 2024-07-01T19:51:49Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/77b2cda57cb3a6a/output.txt

Total script time: 7.87 mins

Integration Tests: Passed

moz-tools-bot · 2024-07-01T20:02:46Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/c5e4786d5ec3077/output.txt

Total script time: 18.81 mins

Integration Tests: FAILED

Snuffleupagus · 2024-07-01T20:32:18Z

web/pdf_viewer.js

+        this.#onPageRenderedCallback = ({ pageNumber }) => {
+          idsToRefresh.delete(pageNumber);
+          if (idsToRefresh.size === 0) {
+            eventBus._off("pagerendered", this.#onPageRenderedCallback);
+            this.#onPageRenderedCallback = null;
+            this.#switchAnnotationEditorModeTimeoutId = setTimeout(updater, 0);
+          }
+        };
+        eventBus._on("pagerendered", this.#onPageRenderedCallback, { signal });


What if the following order of events occur:

The user switches from editing mode NONE to e.g. INK.

The pages are complex enough that re-rendering takes a little while.

The user immediately switches from INK to e.g. STAMP, before rendering has finished.

It seems, unless I'm misunderstanding things, that in such a case there's nothing that'd remove/stop the previously registered "pagerendered" event listener and that one would thus remain forever.
Hence, don't we need to essentially run https://github.com/mozilla/pdf.js/pull/18134/files#diff-4238fc7e96d72f4095859850f3d24a9298ec2dc8bd96437214a02dccbd48385cR2299-R2306 before registering this event listener here?

I think this case is already handled thanks to:
https://github.com/mozilla/pdf.js/pull/18134/files#diff-4238fc7e96d72f4095859850f3d24a9298ec2dc8bd96437214a02dccbd48385cR2315-R2317

We only re-render when we switch to/from NONE. Here in your case we're switch from INK to STAMP.

@calixteman Are you sure, since in the case I'm outlining rendering has not finished before the INK to STAMP switch which means that we've still not updated the this.#annotationEditorMode-field yet and the if you're referencing will thus be true?

Yep you're right: the user should first double click on a freetext and then while waiting double click on a ink... and then we're in the case you mentioned: I'm fixing.

…aining editable annotations Right now, editable annotations are using their own canvas when they're drawn, but it induces several issues: - if the annotation has to be composed with the page then the canvas must be correctly composed with its parent. That means we should move the canvas under canvasWrapper and we should extract composing info from the drawing instructions... Currently it's the case with highlight annotations. - we use some extra memory for those canvas even if the user will never edit them, which the case for example when opening a pdf in Fenix. So with this patch, all the editable annotations are drawn on the canvas. When the user switches to editing mode, then the pages with some editable annotations are redrawn but without them: they'll be replaced by their counterpart in the annotation editor layer.

calixteman · 2024-07-02T12:12:01Z

/botio integrationtest

moz-tools-bot · 2024-07-02T12:12:03Z

From: Bot.io (Linux m4)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/c8812a133dff9a8/output.txt

moz-tools-bot · 2024-07-02T12:12:03Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/80dac6aea917a1d/output.txt

moz-tools-bot · 2024-07-02T12:19:59Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/c8812a133dff9a8/output.txt

Total script time: 7.93 mins

Integration Tests: Passed

moz-tools-bot · 2024-07-02T12:30:28Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/80dac6aea917a1d/output.txt

Total script time: 18.40 mins

Integration Tests: FAILED

calixteman · 2024-07-02T12:39:30Z

/botio-windows integrationtest

moz-tools-bot · 2024-07-02T12:39:34Z

From: Bot.io (Windows)

Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/d465e80d2b9e083/output.txt

moz-tools-bot · 2024-07-02T12:58:10Z

From: Bot.io (Windows)

Success

Full output at http://54.193.163.58:8877/d465e80d2b9e083/output.txt

Total script time: 18.60 mins

Integration Tests: Passed

calixteman added the editor label May 21, 2024

Snuffleupagus changed the title ~~[Editor] When switching to editing mode, redraw pages containing editable annotations~~ [api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations May 21, 2024

calixteman force-pushed the hide_annotations branch from c30d0f8 to 47ab32a Compare May 21, 2024 13:53

calixteman force-pushed the hide_annotations branch from 47ab32a to c19136e Compare May 21, 2024 14:26

Snuffleupagus previously requested changes May 21, 2024

View reviewed changes

calixteman force-pushed the hide_annotations branch 2 times, most recently from 901c6b9 to 2b5eac1 Compare May 29, 2024 08:31

marco-c requested a review from Snuffleupagus May 31, 2024 08:36

marco-c mentioned this pull request May 31, 2024

Unable to select highlighted text. #18190

Open

Snuffleupagus requested changes May 31, 2024

View reviewed changes

calixteman force-pushed the hide_annotations branch 2 times, most recently from bf93f39 to d18df8f Compare May 31, 2024 17:27

marco-c changed the title ~~[api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations~~ [api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) Jun 7, 2024

Snuffleupagus reviewed Jun 7, 2024

View reviewed changes

src/display/api.js Outdated Show resolved Hide resolved

calixteman force-pushed the hide_annotations branch from d18df8f to 5b234c9 Compare June 7, 2024 16:53

Snuffleupagus approved these changes Jun 27, 2024

View reviewed changes

test/integration/test_utils.mjs Outdated Show resolved Hide resolved

test/integration/annotation_spec.mjs Outdated Show resolved Hide resolved

web/pdf_viewer.js Outdated Show resolved Hide resolved

web/pdf_viewer.js Show resolved Hide resolved

calixteman force-pushed the hide_annotations branch from f9d63b4 to dee4005 Compare July 1, 2024 13:37

Snuffleupagus reviewed Jul 1, 2024

View reviewed changes

web/pdf_viewer.js Outdated Show resolved Hide resolved

web/pdf_viewer.js Outdated Show resolved Hide resolved

calixteman force-pushed the hide_annotations branch from dee4005 to 31472a8 Compare July 1, 2024 19:43

Snuffleupagus reviewed Jul 1, 2024

View reviewed changes

calixteman force-pushed the hide_annotations branch from 31472a8 to 64635f3 Compare July 2, 2024 12:11

calixteman merged commit bdcc4a0 into mozilla:master Jul 2, 2024
9 checks passed

calixteman deleted the hide_annotations branch July 2, 2024 13:06

Snuffleupagus mentioned this pull request Jul 3, 2024

Use vertical variant of a char when it's in a missing vertical font (bug 1905623) #18369

Merged

npanchal108 mentioned this pull request Aug 31, 2024

[Snyk] Upgrade pdfjs-dist from 4.0.269 to 4.5.136 npanchal108/ecommercebase#5

Open

[api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) #18134

[api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) #18134

Conversation

calixteman commented May 21, 2024

calixteman commented May 21, 2024

moz-tools-bot commented May 21, 2024

From: Bot.io (Linux m4)

Received

moz-tools-bot commented May 21, 2024

From: Bot.io (Windows)

Received

moz-tools-bot commented May 21, 2024

From: Bot.io (Linux m4)

Failed

moz-tools-bot commented May 21, 2024

From: Bot.io (Windows)

Failed

calixteman commented May 21, 2024

moz-tools-bot commented May 21, 2024

From: Bot.io (Linux m4)

Received

moz-tools-bot commented May 21, 2024

From: Bot.io (Windows)

Received

Snuffleupagus left a comment

Choose a reason for hiding this comment

moz-tools-bot commented May 21, 2024

From: Bot.io (Linux m4)

Failed

moz-tools-bot commented May 21, 2024

From: Bot.io (Windows)

Failed

timvandermeij commented May 21, 2024 • edited Loading

calixteman commented May 29, 2024

Snuffleupagus left a comment • edited Loading

Choose a reason for hiding this comment

calixteman commented May 31, 2024

marco-c commented Jun 7, 2024

calixteman commented Jun 27, 2024

moz-tools-bot commented Jun 27, 2024

From: Bot.io (Linux m4)

Received

moz-tools-bot commented Jun 27, 2024

From: Bot.io (Windows)

Received

moz-tools-bot commented Jun 27, 2024

From: Bot.io (Linux m4)

Success

moz-tools-bot commented Jun 27, 2024

From: Bot.io (Windows)

Failed

Snuffleupagus left a comment • edited Loading

Choose a reason for hiding this comment

timvandermeij commented Jul 1, 2024 • edited Loading

calixteman commented Jul 1, 2024

moz-tools-bot commented Jul 1, 2024

From: Bot.io (Windows)

Received

moz-tools-bot commented Jul 1, 2024

From: Bot.io (Linux m4)

Received

moz-tools-bot commented Jul 1, 2024

From: Bot.io (Linux m4)

Success

moz-tools-bot commented Jul 1, 2024

From: Bot.io (Windows)

Failed

Snuffleupagus Jul 1, 2024 • edited Loading

Choose a reason for hiding this comment

calixteman Jul 1, 2024

Choose a reason for hiding this comment

Snuffleupagus Jul 2, 2024 • edited Loading

Choose a reason for hiding this comment

calixteman Jul 2, 2024

Choose a reason for hiding this comment

calixteman commented Jul 2, 2024

moz-tools-bot commented Jul 2, 2024

From: Bot.io (Linux m4)

Received

moz-tools-bot commented Jul 2, 2024

timvandermeij commented May 21, 2024 •

edited

Loading

Snuffleupagus left a comment •

edited

Loading

Snuffleupagus left a comment •

edited

Loading

timvandermeij commented Jul 1, 2024 •

edited

Loading

Snuffleupagus Jul 1, 2024 •

edited

Loading

Snuffleupagus Jul 2, 2024 •

edited

Loading