Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) #18134

Merged
merged 1 commit into from
Jul 2, 2024

Conversation

calixteman
Copy link
Contributor

Right now, editable annotations are using their own canvas when they're drawn, but it induces several issues:

  • if the annotation has to be composed with the page then the canvas must be correctly composed with its parent. That means we should move the canvas under canvasWrapper and we should extract composing info from the drawing instructions... Currently it's the case with highlight annotations.
  • we use some extra memory for those canvas even if the user will never edit them, which the case for example when opening a pdf in Fenix.

So with this patch, all the editable annotations are drawn on the canvas. When the user switches to editing mode, then the pages with some editable annotations are redrawn but without them: they'll be replaced by their counterpart in the annotation editor layer.

@calixteman
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/848b5ec2a6372b9/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/faccdf624756a1a/output.txt

@Snuffleupagus Snuffleupagus changed the title [Editor] When switching to editing mode, redraw pages containing editable annotations [api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations May 21, 2024
@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Failed

Full output at http://54.241.84.105:8877/848b5ec2a6372b9/output.txt

Total script time: 28.31 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 38
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/848b5ec2a6372b9/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/faccdf624756a1a/output.txt

Total script time: 43.23 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 11

Image differences available at: http://54.193.163.58:8877/faccdf624756a1a/reftest-analyzer.html#web=eq.log

@calixteman
Copy link
Contributor Author

/botio test

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_test from @calixteman received. Current queue size: 1

Live output at: http://54.241.84.105:8877/f5671dedc2e3b62/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_test from @calixteman received. Current queue size: 1

Live output at: http://54.193.163.58:8877/e5817c6d4a755b2/output.txt

Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to review this properly later (probably next week since it's a lot of code), however I unfortunately have several reservations regarding the API so setting r- for now.

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Failed

Full output at http://54.241.84.105:8877/f5671dedc2e3b62/output.txt

Total script time: 28.63 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 36
  different first/second rendering: 2

Image differences available at: http://54.241.84.105:8877/f5671dedc2e3b62/reftest-analyzer.html#web=eq.log

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/e5817c6d4a755b2/output.txt

Total script time: 43.12 mins

  • Unit tests: Passed
  • Integration Tests: FAILED
  • Regression tests: FAILED
  different ref/snapshot: 16

Image differences available at: http://54.193.163.58:8877/e5817c6d4a755b2/reftest-analyzer.html#web=eq.log

@timvandermeij
Copy link
Contributor

timvandermeij commented May 21, 2024

@calixteman Given the size of the patch would it be an idea, to make reviewing easier by reducing the number of changes in this PR, to extract the test refactorings into a separate PR? I'd be happy to review that part and we can then land it first so @Snuffleupagus can check the remainder of the PR later. Those test refactorings (introduction of all switch<mode> helper functions and the two test utility functions) are useful to have in general and make all integration tests consistent which would also help for other integration test PRs such as the waitForTimeout and the Chrome BiDi ones (the more improvements we have in place, the better).

@calixteman calixteman force-pushed the hide_annotations branch 2 times, most recently from 901c6b9 to 2b5eac1 Compare May 29, 2024 08:31
@calixteman
Copy link
Contributor Author

@calixteman Given the size of the patch would it be an idea, to make reviewing easier by reducing the number of changes in this PR, to extract the test refactorings into a separate PR? I'd be happy to review that part and we can then land it first so @Snuffleupagus can check the remainder of the PR later. Those test refactorings (introduction of all switch<mode> helper functions and the two test utility functions) are useful to have in general and make all integration tests consistent which would also help for other integration test PRs such as the waitForTimeout and the Chrome BiDi ones (the more improvements we have in place, the better).

Yep, it has been done in #18165.

Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to say, but I'm finding this somewhat difficult to wrap my head around w.r.t. both the API and viewer (which is why I've not done a proper review yet). Any chance that it could be somehow split into smaller parts and/or simplified somehow?

Given that re-rendering of a page can in some cases be slow, how does this perform when switching to/from editing-mode?

Also, I really don't at all understand how the API works since it seems that you're "mixing" page-specific and global state on the worker-thread; i.e. the EndEditingSession functionality.
Can't this lead to intermittent issues if that changes while a page is parsing/rendering?
To me it seems that the editing-state needs to be constant for a getOperatorList invocation.

There's also the way that the intentState is now handled in the API, adding complexity in the render-method, which suggests to me that the cacheKey isn't correctly computed in the getRenderingIntent-helper.

@calixteman calixteman force-pushed the hide_annotations branch 2 times, most recently from bf93f39 to d18df8f Compare May 31, 2024 17:27
@calixteman
Copy link
Contributor Author

Given that re-rendering of a page can in some cases be slow, how does this perform when switching to/from editing-mode?

If a page with an editable annotation is slow to render then switching to editing mode will be slower.
It behaves like this in Acrobat and I don't really see how we can do better.
The main advantages of this approach are:

  • less memory consumption;
  • better rendering of highlights (when it'll be possible to edit them).

That said if you've any good ideas, as usual please share.

@marco-c marco-c changed the title [api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations [api-minor][Editor] When switching to editing mode, redraw pages containing editable annotations (bug 1883884) Jun 7, 2024
@marco-c
Copy link
Contributor

marco-c commented Jun 7, 2024

This is the first step for bug 1883884.

src/display/api.js Outdated Show resolved Hide resolved
@calixteman
Copy link
Contributor Author

/botio integrationtest

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/9cab3b8d503873c/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/c13dc56b765177d/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/9cab3b8d503873c/output.txt

Total script time: 7.77 mins

  • Integration Tests: Passed

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/c13dc56b765177d/output.txt

Total script time: 18.41 mins

  • Integration Tests: FAILED

Copy link
Collaborator

@Snuffleupagus Snuffleupagus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: Given that we have a pending release (hopefully soon), we should probably not land this PR just prior to that.

I've not tested this very much, so I'm counting on you having tested this thoroughly during development :-)
Also, the code feels a lot more approachable now compared to the initial version; thanks for that!

r=me, with a couple of final comments addressed; thank you!

test/integration/test_utils.mjs Outdated Show resolved Hide resolved
test/integration/annotation_spec.mjs Outdated Show resolved Hide resolved
web/pdf_viewer.js Outdated Show resolved Hide resolved
web/pdf_viewer.js Show resolved Hide resolved
web/pdf_viewer.js Outdated Show resolved Hide resolved
web/pdf_viewer.js Outdated Show resolved Hide resolved
@timvandermeij
Copy link
Contributor

timvandermeij commented Jul 1, 2024

Moreover, since this will probably land as the first [api-minor] change after the new release, please also bump the library version in a separate PR after merging this (so there is a merge commit) similar to e.g. 648a2d8.

@calixteman
Copy link
Contributor Author

/botio integrationtest

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/c5e4786d5ec3077/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/77b2cda57cb3a6a/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/77b2cda57cb3a6a/output.txt

Total script time: 7.87 mins

  • Integration Tests: Passed

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/c5e4786d5ec3077/output.txt

Total script time: 18.81 mins

  • Integration Tests: FAILED

Comment on lines +2333 to +2341
this.#onPageRenderedCallback = ({ pageNumber }) => {
idsToRefresh.delete(pageNumber);
if (idsToRefresh.size === 0) {
eventBus._off("pagerendered", this.#onPageRenderedCallback);
this.#onPageRenderedCallback = null;
this.#switchAnnotationEditorModeTimeoutId = setTimeout(updater, 0);
}
};
eventBus._on("pagerendered", this.#onPageRenderedCallback, { signal });
Copy link
Collaborator

@Snuffleupagus Snuffleupagus Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the following order of events occur:

  • The user switches from editing mode NONE to e.g. INK.
  • The pages are complex enough that re-rendering takes a little while.
  • The user immediately switches from INK to e.g. STAMP, before rendering has finished.

It seems, unless I'm misunderstanding things, that in such a case there's nothing that'd remove/stop the previously registered "pagerendered" event listener and that one would thus remain forever.
Hence, don't we need to essentially run https://github.com/mozilla/pdf.js/pull/18134/files#diff-4238fc7e96d72f4095859850f3d24a9298ec2dc8bd96437214a02dccbd48385cR2299-R2306 before registering this event listener here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this case is already handled thanks to:
https://github.com/mozilla/pdf.js/pull/18134/files#diff-4238fc7e96d72f4095859850f3d24a9298ec2dc8bd96437214a02dccbd48385cR2315-R2317

We only re-render when we switch to/from NONE. Here in your case we're switch from INK to STAMP.

Copy link
Collaborator

@Snuffleupagus Snuffleupagus Jul 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@calixteman Are you sure, since in the case I'm outlining rendering has not finished before the INK to STAMP switch which means that we've still not updated the this.#annotationEditorMode-field yet and the if you're referencing will thus be true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep you're right: the user should first double click on a freetext and then while waiting double click on a ink... and then we're in the case you mentioned: I'm fixing.

…aining editable annotations

Right now, editable annotations are using their own canvas when they're drawn, but
it induces several issues:
 - if the annotation has to be composed with the page then the canvas must be correctly
   composed with its parent. That means we should move the canvas under canvasWrapper
   and we should extract composing info from the drawing instructions...
   Currently it's the case with highlight annotations.
 - we use some extra memory for those canvas even if the user will never edit them, which
   the case for example when opening a pdf in Fenix.

So with this patch, all the editable annotations are drawn on the canvas. When the
user switches to editing mode, then the pages with some editable annotations are redrawn but
without them: they'll be replaced by their counterpart in the annotation editor layer.
@calixteman
Copy link
Contributor Author

/botio integrationtest

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/c8812a133dff9a8/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/80dac6aea917a1d/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Linux m4)


Success

Full output at http://54.241.84.105:8877/c8812a133dff9a8/output.txt

Total script time: 7.93 mins

  • Integration Tests: Passed

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Failed

Full output at http://54.193.163.58:8877/80dac6aea917a1d/output.txt

Total script time: 18.40 mins

  • Integration Tests: FAILED

@calixteman
Copy link
Contributor Author

/botio-windows integrationtest

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Received

Command cmd_integrationtest from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/d465e80d2b9e083/output.txt

@moz-tools-bot
Copy link
Collaborator

From: Bot.io (Windows)


Success

Full output at http://54.193.163.58:8877/d465e80d2b9e083/output.txt

Total script time: 18.60 mins

  • Integration Tests: Passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants