Intermittent bug with XFA test on the bots

[calixteman]

I ran makeref on the linux bot using X11 forwarding and I managed to get a profile when Firefox hung. For information, I found nothing neither in the console nor in the terminal nor in network panel which helped me to guess what was wrong.
I added few console.log to try to figure out where the hang happened and it was during toHTML:

pdf.js/src/core/xfa/factory.js

Line 43 in 064c21d

this.pages = this.form[$toHTML]();

and so I finally added a setTimeout(() => console.log(node.constructor.name), 10000) + a clearTimeout around the call to toHTML: https://github.com/mozilla/pdf.js/blob/master/src/core/xfa/xfa_object.js#L370 (and the same around the toHTML in template.js file).
And I ran makeref in a while true during 8 hours and no hang... :(

About the profile, from @gabrielesvelto: "Firefox isn't doing nothing but it's waiting for an event in main event loop".

[calixteman]

@Snuffleupagus, @timvandermeij, if you've any idea to try something which could help to figure out what's wrong please propose.

[Snuffleupagus]

One thing that I've noticed is different between XFA documents and "regular" documents, is that the XFA documents are parsed all once. As far as I understand the data structure of XFA documents basically force this, however I do wonder if this is maybe somehow related to these problems!?

Note how for "regular" documents, during both parsing and rendering (see src/core/evaluator.js and src/display/canvas.js), we'll divide the parsing/rendering into small chunks and repeatedly defer to the event loop to avoid long running parsing/rendering from hanging the browser.

Hence my, possibly dumb, idea: Would it be possible to make the XFA parsing slightly asynchronous, for example by deferring to the event loop after each parsed page?

[marco-c]

We could try to go back in the history of PRs and try to figure out what was the first change where we hit this failure.

[brendandahl]

Note: the linux bot IP changed during this time, but the logs should still be on the server. new ip 54.241.84.105

[calixteman]

I think the first time we saw the problem was in this PR:
#13653

[calixteman]

@Snuffleupagus, I don't understand why you could be right but it's definitely something I want to try.
So I think it should be possible to do that in using transforming:

pdf.js/src/core/xfa/template.js

Line 5398 in 6a15973

[$toHTML]() {

into a generator function to yield each page which could allow us to delay the conversion for the next page in using a setTimeout. Having a generator function would help to save and restore the state we were before yielding a page.