Flaky test

[lishaduck]

Moving this thread so I don't lose track of it: ↓

I've noticed this there's a test that's been flaking ("Running using other configuration (without errors)"):

-- UNEXPECTED ERROR ------------------------------------------------------------

I ran into an unexpected error. Please open an issue at the following link:
  https://github.com/jfmengels/node-elm-review/issues/new

Please include this error message and as much detail as you can provide. Running
with --debug might give additional information. If you can, please provide a
setup that makes it easy to reproduce the error. That will make it much easier
to fix the issue.

Below is the error that was encountered.
--------------------------------------------------------------------------------
<local-path>/test/project-with-errors/elm-stuff/generated-code/jfmengels/elm-review/cli/<version>/review-applications/bd2a2bb6c965dac5241ce8c4d0615d83.js:10328
    		$temp$offset = offset + 
    						                    

SyntaxError: Unexpected end of input

  at loadCompiledElmApp (<local-path>/lib/load-compiled-app.js:28:18)
  at initWithoutWorker (<local-path>/lib/app-wrapper.js:144:21)
  "
  at Object.test (test/review.test.js:37:1)

Originally posted by @lishaduck in #308 (comment)

[lishaduck]

I've noticed this there's a test that's been flaking ("Running using other configuration (without errors)"):

Yes, I've noticed the same thing. I had a look, thinking it could be us not waiting for the compilation to be finalized, but the code looks fine. I still think it's the writing of the file (or the loading of it, but that's weirder) that is getting cut off early somehow, but I haven't figured out how yet. 🤔

It sounds pretty recent though. I don't think we saw this one before, but I could be wrong.

Originally posted by @jfmengels in #308 (comment)

[lishaduck]

It sounds pretty recent though. I don't think we saw this one before, but I could be wrong.

Well, it's really started flaking. Will look into it.
For reference, here's the first failure from it: https://github.com/jfmengels/node-elm-review/actions/runs/11769034390/job/32779541170

EDIT: Commit-wise, there was a different flake on #306 (it's older than my work here, I think I remember seeing it a while ago), and then fs-extra (#303) seems to have started this flake.

EDIT 2: Am gonna try to see if I reproduce it before #303.

Originally posted by @lishaduck in #308 (comment)

[lishaduck]

Was able to repro with 38453c1, so sadly not #303.

Originally posted by @lishaduck in #308 (comment)

[lishaduck]

Huh. I can't repro on #304 for the life of me,¹ so I guess it's gotta be #306 (or, I suppose, #305), which makes no sense.

Originally posted by @lishaduck in #308 (comment)

Footnotes

1. I've run tests 10+ 9 times on the commit, to no avail. ↩

[lishaduck]

@jfmengels, any ideas?

Originally posted by @lishaduck in #308 (comment)

[jfmengels]

I haven't had the time to delve into it more, so my thoughts are still the same as what is written above.

Do you have the power to re-run CI runs by the way or is it only me? (Because if you can, then this is annoying but more easily bypassable).

[lishaduck]

Do you have the power to re-run CI runs by the way or is it only me? (Because if you can, then this is annoying but more easily bypassable).

No, I don't unfortunately (surprisingly, GitHub has a single secure default for actions 🎉 🤣).

[lishaduck]

What? Another flake:
https://github.com/jfmengels/node-elm-review/actions/runs/11902573547/job/33167861766

This time in "should retrieve elm binary from PATH":

"-- UNEXPECTED ERROR ------------------------------------------------------------

I ran into an unexpected error. Please open an issue at the following link:
  https://github.com/jfmengels/node-elm-review/issues/new

Please include this error message and as much detail as you can provide. Running
with --debug might give additional information. If you can, please provide a
setup that makes it easy to reproduce the error. That will make it much easier
to fix the issue.

Below is the error that was encountered.
--------------------------------------------------------------------------------
TypeError: Cannot read properties of undefined (reading 'Elm')

  at initWithoutWorker (<local-path>/lib/app-wrapper.js:[145](https://github.com/jfmengels/node-elm-review/actions/runs/11902573547/job/33167861766#step:10:152):29)
  at Object.init (<local-path>/lib/app-wrapper.js:25:10)
  at Object.initializeApp (<local-path>/lib/runner.js:112:26)
  at <local-path>/lib/main.js:81:27
      at async Promise.all (index 0)
  at async runElmReview (<local-path>/lib/main.js:79:19)
  at async main (<local-path>/lib/main.js:271:5)
  "
  at Object.test (test/compiler-flag.test.js:13:1)

I just noticed that we've got two stacktraces here. Will need to keep that in mind.

It makes me think of 8fbe6f6, but that didn't change any runtime code.

[lishaduck]

I'll try to get to this this week. This was frustrating me, so I took a break to start a "small" side project to help make TS more Elmy, but the scope of that is frustrating me as well, so I'm back :)

[lishaduck]

Oh, fixed by #322 (hopefully).

[lishaduck]

Oh, reopening this as we hit this again, unfortunately. It is a bit better though!

[lishaduck]

Dagumit. Even after #325, it's still running into issues.
Perhaps it's a stale cache in ci? It shouldn't invoking elm-review in parallel anymore. (Wait, does Jest run in parallel? yup).

[jfmengels]

I've cleared the cache just in case 🤷

[lishaduck]

I've cleared the cache just in case 🤷

Can you rerun https://github.com/jfmengels/node-elm-review/actions/runs/12214733248 to see if it helped?

[jfmengels]

It passed this time.

[lishaduck]

Hopefully that was it 🤞

[lishaduck]

Nope, flaked on #317.